CANVASHIL

HUMAN-IN-THE-LOOP • CANVAS • IMPLEMENTATION • AI UIs

INTERACTIVE AI EXPERIENCES

In today's conversational AI landscape, the most powerful applications go beyond simple text exchanges. They create rich, interactive experiences that blend AI capabilities with human input and visual components. This guide introduces you to the Canvas pattern - a powerful approach for building dynamic, interactive spaces within chat-based AI applications.

The Canvas approach transforms conversational interfaces from linear exchanges into rich workspaces where users and AI Agents can solve problems together.

WHAT IS CANVAS IN AI UIs

Canvas is a specialized UI component that provides a dedicated workspace alongside your chat interface. Unlike the standard back-and-forth of chat, a Canvas creates a persistent, interactive space for complex visualizations, user inputs, and dynamic content.

Live examples:

  • 1
    ChatGPT's Canvas - Interactive workspace to write & code with GPT
  • 2
    Gemini's Canvas - Interactive space to collaborate with Gemini
  • 3
    Claude's Artifacts - Standalone content display alongside conversation
  • 4
    LangChain's OpenCanvas - Open-source implementation of OpenAI Canvas

This Guide Covers

  • Canvas UI pattern implementation
  • Human-in-the-Loop with LangGraph interrupt
  • Practical examples across domains
  • Design patterns for interactive experiences

Why Canvas Matters

The Canvas UI pattern enables entirely new kinds of AI applications with richer information display, structured input collection, collaborative workflows, and context preservation throughout conversations.

Visualizations

User Controls

Structured Data

Interactive Media

Pull Quote

"The combination of Canvas UI and LangGraph's interrupt capabilities enables entirely new kinds of applications where humans and AI can truly collaborate."

CORE CONCEPTS

Canvas Pattern

The Canvas pattern represents a complementary workspace alongside a chat interface. Key characteristics include:

  • Persistent Display: Unlike chat messages, canvas content remains visible and can be updated
  • Rich Content: Supports visualizations, forms, interactive elements, and media
  • Complementary to Chat: Works alongside rather than replacing the conversation
  • Stateful: Maintains its state independent of the chat history

Canvas implementations typically feature toggle controls to show/hide the canvas and can be updated by either user actions or agent responses.

Human-in-the-Loop & LangGraph

LangGraph's interrupt mechanism creates a collaborative approach beyond traditional LLM applications:

  • Agent-Initiated Requests: The agent can proactively request specific user input during execution
  • Structured Input Gathering: Request specific data formats or choices rather than free-text responses
  • Workflow Continuation: Resume processing exactly where it left off after receiving user input

┌─────────────┐     ┌───────────────┐     ┌────────────────────┐
│ User Request│────▶│ Agent Process │────▶│ Generate Response  │
└─────────────┘     └───────────────┘     └────────────────────┘
                           │                         ▲
                           ▼                         │
                    ┌────────────┐          ┌────────────────┐
                    │  Interrupt │─────────▶│ Human Response │
                    └────────────┘          └────────────────┘
                            

Combining Canvas & Interrupts

The real power comes from combining these approaches:

  1. Agent identifies a need for specific user input
  2. Interrupts its process and requests input via a specialized UI in the canvas
  3. User provides structured input through the canvas interface
  4. Agent resumes processing with the exact data it needs

This creates a structured, collaborative workflow between human and AI Agents.

USE CASES

The Canvas + Interrupt pattern enables diverse applications across domains:

Travel Planning

✈️

Interactive destination selection and itinerary building

CANVAS CONTENT

Maps, hotel comparisons, activity scheduling

INTERRUPT FOR

Destination selection, date picking, preference collection

VALUE

Visual planning, immediate feedback, personalized recommendations

Fashion & E-commerce

👗

Virtual try-on and styling sessions

CANVAS CONTENT

Virtual avatar, clothing options, style boards

INTERRUPT FOR

Style preference collection, size selection, outfit feedback

VALUE

Visual outfit previews, immediate style feedback, personalized recommendations

Education & Learning

🎓

Interactive tutoring with shared workspace

CANVAS CONTENT

Math problems, diagrams, coding environment, notebook

INTERRUPT FOR

Problem-solving steps, concept understanding checks, practice exercises

VALUE

Step-by-step guidance, visual explanations, hands-on practice

Data Analysis

📊

Collaborative data exploration

CANVAS CONTENT

Charts, data tables, filtering controls

INTERRUPT FOR

Data source selection, visualization preferences, analysis focus

VALUE

Visual data exploration, iterative analysis, insight generation

Healthcare & Diagnostics

🏥

Symptom assessment and health monitoring

CANVAS CONTENT

Body diagrams, symptom trackers, medical visuals

INTERRUPT FOR

Symptom location selection, severity assessment, history collection

VALUE

Precise symptom localization, visual health tracking, educational diagrams

Product Design

🎨

Collaborative design sessions

CANVAS CONTENT

Product mockups, design tools, feedback mechanisms

INTERRUPT FOR

Design preference selection, feature prioritization, feedback collection

VALUE

Visual design iteration, collaborative creation, structured feedback

Building something cool? Let's chat about your implementation.

ARCHITECTURE OVERVIEW

System Architecture


┌────────────────────────────────────────┐
│ UI Layer                               │
│  ┌──────────┐          ┌─────────────┐ │
│  │   Chat   │◄────────►│   Canvas    │ │
│  └──────────┘          └─────────────┘ │
└────────────┬───────────────┬───────────┘
             │               │
             ▼               ▼
┌────────────────┐   ┌──────────────────┐
│ Message Handler│   │ Interrupt Handler│
└────────┬───────┘   └────────┬─────────┘
         │                    │
         ▼                    ▼
┌────────────────────────────────────────┐
│ LangGraph Runtime                      │
│                                        │
│  ┌───────────┐      ┌───────────────┐  │
│  │   Agent   │◄────►│  Interrupts   │  │
│  └───────────┘      └───────────────┘  │
└────────────────────────────────────────┘
                        

Key Components

  • Thread.tsx: Manages the chat interface and message display
  • Canvas.tsx: Implements the canvas UI component
  • InterruptHandler.tsx: Routes different interrupt types
  • Specific Interrupt Components: Specialized UIs for different interrupt types
  • LangGraph SDK: Communicates with the LangGraph server

Data Flow

  1. 1
    User sends a message through the chat interface
  2. 2
    Message is sent to the LangGraph server
  3. 3
    Agent processes the message and may interrupt for user input
  4. 4
    Frontend detects the interrupt and renders the canvas
  5. 5
    User interacts with the canvas, providing input
  6. 6
    Input is sent back to the agent
  7. 7
    Agent continues processing and completes the task

IMPLEMENTATION HIGHLIGHTS

Key Implementation Concepts

Canvas Component Approach

The Canvas implementation uses a compound component pattern with distinct parts (Root, Header, Content, Footer) that work together. This provides flexibility and consistent structure across different canvas uses.

Reference: See components/ui/canvas.tsx in the source code

Interrupt Handler Pattern

The interrupt handler uses a type-based routing pattern to direct different interrupt types to specialized components. This keeps the code modular and makes adding new interrupt types straightforward.

Reference: See components/interrupt-handler.tsx in the source code

State Management Approach

Canvas state is managed through a dedicated context provider and hook system, making it available throughout the application while keeping the implementation details abstracted away from consumer components.

Reference: See context/canvas-context.tsx and hooks/use-canvas.tsx in the source code

Chat UI

For handling chat-specific features like streaming, auto-scrolling, and markdown rendering, libraries such as Assistant UI are great at managing the chat while integrating with canvas implementations.

All implementation code for Canvas and Human-in-the-Loop integration is available in the open-source repository. I encourage you to explore the code for detailed implementation approaches.

PRO TIPS

Progressive Disclosure

Start simple and progressively reveal more options as needed to avoid overwhelming users with too many options or complex interfaces.

Pattern: Show a simplified interface first, with an option to expand for advanced features.

Contextual Canvas

Adapt canvas content to the conversation context by using the agent state to determine what to show in the canvas.

Pattern: Switch between different canvas views based on the current topic or agent state.

Interrupt Chaining

Chain interrupts together to collect multiple pieces of information sequentially, with each interrupt triggering the next.

Pattern: Use state in the agent to track what information is still needed and interrupt accordingly.

Two-Way Visualization

Create visualizations that can be updated by both user inputs and AI outputs to visually represent the collaborative work.

Pattern: Implement clear visual distinctions between user and AI contributions in shared visualizations.

CONCLUSION

The combination of Canvas UI and LangGraph interrupts creates powerful new possibilities for collaborative AI applications. By implementing these patterns, you can build experiences where:

Collaboration

Users and AI truly work together rather than just exchanging messages

Visualization

Complex information is visualized in dedicated spaces

Structure

Input collection is structured and context-aware

Flow Control

Workflows seamlessly transition between AI and human control

This guide has provided the tools, patterns, and examples needed to implement these capabilities in your own applications. As you build with these techniques, you'll discover even more ways to enhance human-AI collaboration and create truly interactive experiences.

Have questions about your implementation?

I'm available to help you working with Canvas, AI UIs and Agents.