Smoke Test

DeerFlow Smoke Test Skill

This skill guides the Agent through DeerFlow's full end-to-end smoke test workflow, including code updates, deployment (supporting both Docker and local installation modes), service availability verification, and health checks.

Deployment Mode Selection

This skill supports two deployment modes:

• Local installation mode (recommended, especially when network issues occur) - Run all services directly on the local machine
• Docker mode - Run all services inside Docker containers

Selection strategy:

• If the user explicitly asks for Docker mode, use Docker
• If network issues occur (such as slow image pulls), automatically switch to local mode
• Default to local mode whenever possible

Structure

smoke-test/
├── SKILL.md                          ← You are here - core workflow and logic
├── scripts/
│   ├── check_docker.sh               ← Check the Docker environment
│   ├── check_local_env.sh            ← Check local environment dependencies
│   ├── frontend_check.sh             ← Frontend page smoke check
│   ├── pull_code.sh                  ← Pull the latest code
│   ├── deploy_docker.sh              ← Docker deployment
│   ├── deploy_local.sh               ← Local deployment
│   └── health_check.sh               ← Service health check
├── references/
│   ├── SOP.md                        ← Standard operating procedure
│   └── troubleshooting.md            ← Troubleshooting guide
└── templates/
    ├── report.local.template.md      ← Local mode smoke test report template
    └── report.docker.template.md     ← Docker mode smoke test report template

Standard Operating Procedure (SOP)

Phase 1: Code Update Check

1. Confirm current directory - Verify that the current working directory is the DeerFlow project root
2. Check Git status - See whether there are uncommitted changes
3. Pull the latest code - Use git pull origin main to get the latest updates
4. Confirm code update - Verify that the latest code was pulled successfully

Phase 2: Deployment Mode Selection and Environment Check

Choose deployment mode:

• Ask for user preference, or choose automatically based on network conditions
• Default to local installation mode

Local mode environment check:

1. Check Node.js version - Requires 22+
2. Check pnpm - Package manager
3. Check uv - Python package manager
4. Check nginx - Reverse proxy
5. Check required ports - Confirm that ports 2026, 3000, and 8001 are not occupied

Docker mode environment check (if Docker is selected):

1. Check whether Docker is installed - Run docker --version
2. Check Docker daemon status - Run docker info
3. Check Docker Compose availability - Run docker compose version
4. Check required ports - Confirm that port 2026 is not occupied

Phase 3: Configuration Preparation

1. Check whether config.yaml exists
   • If it does not exist, run make config to generate it
   • If it already exists, check whether it needs an upgrade with make config-upgrade
2. Check the .env file
   • Verify that required environment variables are configured
   • Especially model API keys such as OPENAI_API_KEY

Phase 4: Deployment Execution

Local mode deployment:

1. Check dependencies - Run make check
2. Install dependencies - Run make install
3. (Optional) Pre-pull the sandbox image - If needed, run make setup-sandbox
4. Start services - Run make start (production mode)
5. Wait for startup - Give all services enough time to start completely (90-120 seconds recommended)

Docker mode deployment (if Docker is selected):

1. Initialize Docker environment - Run make docker-init
2. Start Docker services - Run make up (production mode)
3. Wait for startup - Give all containers enough time to start completely (60 seconds recommended)

Phase 5: Service Health Check

Local mode health check:

1. Check process status - Confirm that Gateway, Frontend, and Nginx processes are all running
2. Check frontend service - Visit http://localhost:2026 and verify that the page loads
3. Check API Gateway - Verify the http://localhost:2026/health endpoint
4. Check LangGraph-compatible API - Verify the /api/langgraph/* route exposed by Gateway
5. Frontend route smoke check - Run bash .agent/skills/smoke-test/scripts/frontend_check.sh to verify key routes under /workspace. The script auto-detects whether authentication is enabled and, if so, registers / logs in a smoke-test user so the real pages are verified rather than the login redirect.

Docker mode health check (when using Docker):

1. Check container status - Run docker ps and confirm that all containers are running
2. Check frontend service - Visit http://localhost:2026 and verify that the page loads
3. Check API Gateway - Verify the http://localhost:2026/health endpoint
4. Check LangGraph-compatible API - Verify the /api/langgraph/* route exposed by Gateway
5. Frontend route smoke check - Run bash .agent/skills/smoke-test/scripts/frontend_check.sh to verify key routes under /workspace. The script auto-detects whether authentication is enabled and, if so, registers / logs in a smoke-test user so the real pages are verified rather than the login redirect.

Optional Functional Verification

1. List available models - Verify that model configuration loads correctly
2. List available skills - Verify that the skill directory is mounted correctly
3. Simple chat test - Send a simple message to verify the end-to-end flow

Phase 6: Generate Test Report

1. Collect all test results - Summarize execution status for each phase
2. Record encountered issues - If anything fails, record the error details
3. Generate the final report - Use the template that matches the selected deployment mode to create the complete test report, including overall conclusion, detailed key test cases, and explicit frontend page / route results
4. Provide follow-up recommendations - Offer suggestions based on the test results

Execution Rules

• Follow the sequence - Execute strictly in the order described above
• Idempotency - Every step should be safe to repeat
• Error handling - If a step fails, stop and report the issue, then provide troubleshooting suggestions
• Detailed logging - Record the execution result and status of each step
• User confirmation - Ask for confirmation before potentially risky operations such as overwriting config
• Mode preference - Prefer local mode to avoid network-related issues
• Template requirement - The final report must use the matching template under templates/; do not output a free-form summary instead of the template-based report
• Report clarity - The execution summary must include the overall pass/fail conclusion plus per-case result explanations, and frontend smoke check results must be listed explicitly in the report
• Optional phase handling - If functional verification is not executed, do not present it as a separate skipped phase in the final report

Known Acceptable Warnings

The following warnings can appear during smoke testing and do not block a successful result:

• Feishu/Lark SSL errors in Gateway logs (certificate verification failure) can be ignored if that channel is not enabled
• Warnings in Gateway logs about missing methods in the custom checkpointer, such as adelete_for_runs or aprune, do not affect the core functionality
• The frontend_check.sh script automatically handles authentication. When auth is enabled it registers / logs in a smoke-test user (smoke-test@deerflow.dev by default) to verify the real /workspace/* pages. The registration may produce a log entry from the auth provider, which is expected and harmless.

Key Tools

Use the following tools during execution:

1. bash - Run shell commands
2. present_file - Show generated reports and important files
3. task_tool - Organize complex steps with subtasks when needed

Success Criteria

Smoke test pass criteria (local mode):

• Latest code is pulled successfully
• Local environment check passes (Node.js 22+, pnpm, uv, nginx)
• Configuration files are set up correctly
• make check passes
• make install completes successfully
• make dev starts successfully
• All service processes run normally
• Frontend page is accessible
• Frontend route smoke check passes (/workspace key routes)
• API Gateway health check passes
• Test report is generated completely

Smoke test pass criteria (Docker mode):

• Latest code is pulled successfully
• Docker environment check passes
• Configuration files are set up correctly
• make docker-init completes successfully
• make up completes successfully
• All Docker containers run normally
• Frontend page is accessible
• Frontend route smoke check passes (/workspace key routes)
• API Gateway health check passes
• Test report is generated completely

Read Reference Files

Before starting execution, read the following reference files:

1. references/SOP.md - Detailed step-by-step operating instructions
2. references/troubleshooting.md - Common issues and solutions
3. templates/report.local.template.md - Local mode test report template
4. templates/report.docker.template.md - Docker mode test report template