2D trajectories

  • Visualizing LLM Thinking with Python Toolkit


    [Project] I treated LLM inference like a physical signal trajectory. Here is a Python toolkit to visualize the "Thinking Process" (Hidden States).A PhD student in Electromagnetics developed a Python toolkit to visualize the "thinking process" of Local LLMs by treating inference as a physical signal trajectory. This tool extracts hidden states layer-by-layer and presents them as 2D/3D trajectories, revealing insights such as the "Confidence Funnel," where different prompts converge into a single attractor basin, and distinct "Thinking Styles" between models like Llama-3 and Qwen-2.5. Additionally, the toolkit visualizes model behaviors like "Refusal" during safety checks, offering a geometric perspective on model dynamics and safety tuning. This approach provides a novel way to profile model behaviors beyond traditional benchmarks.

    Read Full Article: Visualizing LLM Thinking with Python Toolkit