• MLSys @ WukLab - Can Scheduling Overhead Dominate LLM Inference Performance? A Study of CPU Scheduling Overhead on Two Popular LLM Inference Systems

  15 min read   ·   October 14, 2024   ·   WukLab

  2024   ·   llm-inference

• MLSys @ WukLab - Preble: Efficient Prompt Scheduling for Augmented Large Language Models

  7 min read   ·   May 14, 2024   ·   WukLab

  2024   ·   llm-inference