# How CI/CD Execution Works

# Docker-Based Test Execution

All CI/CD test execution in KushoAI is powered by Docker containers, providing consistent and reliable test execution across all supported platforms. The Docker test runner handles various execution modes and integrates seamlessly with your existing CI/CD workflows.

# Docker Image

The Kusho test runner is available as a public Docker image:

public.ecr.aws/y5g4u6y7/kusho-test-runner:latest

This image contains all necessary dependencies and tools to execute API tests and E2E workflows in any Docker-compatible environment.

# Execution Methods

# 1. API Test Suite Execution by UUID

Execute specific test suites using their unique identifiers:

docker run --rm \
  -e BASE_URL="https://be.kusho.ai" \
  -e ENVIRONMENT_ID="2" \
  -e API_KEY="your-api-key" \
  -e CI_COMMIT_SHA="commit-hash" \
  -e CI_COMMIT_MESSAGE="commit message" \
  -e TEST_SUITE_UUID="uuid1,uuid2,uuid3" \
  public.ecr.aws/y5g4u6y7/kusho-test-runner:latest

Use Cases:

Testing specific APIs or endpoints
Targeted testing for specific features
Running critical path tests

# 2. API Test Suite Execution by Group ID

Execute all test suites within a specific group:

docker run --rm \
  -e BASE_URL="https://be.kusho.ai" \
  -e ENVIRONMENT_ID="2" \
  -e API_KEY="your-api-key" \
  -e CI_COMMIT_SHA="commit-hash" \
  -e CI_COMMIT_MESSAGE="commit message" \
  -e GROUP_ID="87" \
  public.ecr.aws/y5g4u6y7/kusho-test-runner:latest

Use Cases:

Running all tests for a specific service or module
Organized test execution by feature area
Team-based test organization

# 3. API Test Suite Execution by Tags

Execute test suites filtered by specific tags:

docker run --rm \
  -e BASE_URL="https://be.kusho.ai" \
  -e ENVIRONMENT_ID="2" \
  -e API_KEY="your-api-key" \
  -e CI_COMMIT_SHA="commit-hash" \
  -e CI_COMMIT_MESSAGE="commit message" \
  -e TAGS="smoke,regression,critical" \
  public.ecr.aws/y5g4u6y7/kusho-test-runner:latest

Use Cases:

Running smoke tests before deployment
Executing regression tests after major changes
Targeted testing based on test characteristics

# 4. E2E Workflow Execution

Execute end-to-end workflows with execution profiles:

docker run --rm \
  -e BASE_URL="https://be.kusho.ai" \
  -e ENVIRONMENT_ID="2" \
  -e API_KEY="your-api-key" \
  -e CI_COMMIT_SHA="commit-hash" \
  -e CI_COMMIT_MESSAGE="E2E testing" \
  -e E2E_TEST_SUITE_UUID="e5d1b703-a798-4bd9-becb-8977bc2a1ec3" \
  -e EXECUTION_PROFILE_UUID="execution-profile-uuid" \
  public.ecr.aws/y5g4u6y7/kusho-test-runner:latest

Use Cases:

Testing complete user journeys
Validating API integrations and data flow
Comprehensive workflow testing

# Environment Variables Reference

# Required Variables

Variable	Description	Example
`API_KEY`	Authentication key for Kusho API	`"api-key-dadsadsadsadsaasa"`
`ENVIRONMENT_ID`	Target environment ID	`"2"`

# Optional Variables

Variable	Description	Default	Example
`BASE_URL`	Kusho API base URL	`"https://be.kusho.ai"`	`"https://on-prem-kusho-backend.yourdomain.com"`
`EMAIL`	Email for notifications	None	`"user@example.com"`
`CI_COMMIT_SHA`	Git commit hash	None	`"a1b2c3d4"`
`CI_COMMIT_MESSAGE`	Git commit message	None	`"Fix user authentication"`

# Execution Mode Variables (One Required)

Variable	Description	Example
`TEST_SUITE_UUID`	Comma-separated test suite UUIDs	`"uuid1,uuid2"`
`GROUP_ID`	Test suite group ID	`"87"`
`TAGS`	Comma-separated tags	`"smoke,regression"`
`E2E_TEST_SUITE_UUID` + `EXECUTION_PROFILE_UUID`	E2E workflow and profile UUIDs	See E2E example above

# Output and Exit Codes

# Test Result Output

The test runner provides detailed output for different execution modes:

API Test Suite Output:

Test case descriptions
HTTP status codes with color coding
Test execution status (PASS/FAIL)
Assertion results

E2E Test Suite Output:

Workflow information and progress
Combination execution details
Individual test case results with color coding:
- Status Codes: Green (2xx), Yellow (4xx), Red (5xx), Gray (N/A)
- Assertions: Green (PASS), Red (FAIL), Gray (N/A)

# Exit Codes

0: All tests passed successfully
1: One or more tests failed or execution error occurred

These exit codes integrate with CI/CD systems to determine build success or failure.

# E2E Workflow Execution Details

E2E test suites execute test cases in combinations based on workflow structure:

Workflow Graph: Defines the sequence of test suites connected by data flow
Execution Profile: Specifies which test cases to run from each test suite (refer this section for more details)
Cartesian Product: Creates all possible combinations of test cases across connected test suites

# Example E2E Execution

For a workflow: TestSuite1 (2 test cases) → TestSuite2 (3 test cases)

The system generates 6 execution combinations:

TS1-TC1 → TS2-TC1
TS1-TC1 → TS2-TC2
TS1-TC1 → TS2-TC3
TS1-TC2 → TS2-TC1
TS1-TC2 → TS2-TC2
TS1-TC2 → TS2-TC3

Each combination represents a complete execution path through the workflow with proper data flow validation.

# Best Practices

# Security

Store sensitive data (API keys) as platform secrets, never in code
Use environment-specific configurations
Implement proper access controls for test execution

# Performance

Use appropriate test selection (UUID, Group, Tags) to minimize execution time
Run critical tests early in pipelines for fast feedback
Implement parallel execution where supported

# Monitoring

Set up notifications for test failures
Track test execution trends over time
Monitor test execution duration and success rates

# Organization

Use descriptive commit messages for better test result tracking
Organize tests using groups and tags for flexible execution
Maintain execution profiles for consistent E2E testing