Actions

An Action is a named sequence of steps that process data. Actions are the building blocks of your pipeline logic.

Basic structure

action ActionName {
  // Step 1
  get "/endpoint"

  // Step 2
  for item in response {
    // Nested steps
  }

  // Step 3
  store response -> storeName { key: .id }
}

Step types

Actions can contain the following step types:

Step	Description
get, post, put, patch, delete	HTTP requests
call	OAS operation call
for...in...where	Iteration with optional filtering
map...->	Data transformation
validate	Constraint checking
store...->	Data persistence
match	Pattern matching with flow control

HTTP request steps

Fetch data from APIs:

action FetchData {
  // Simple GET
  get "/users"

  // With query parameters
  get "/users" { params: { limit: 100, offset: 0 } }

  // With pagination
  get "/users" {
    paginate: offset(page, 100),
    until: length(response.users) == 0
  }

  // POST with body
  post "/users" {
    body: {
      name: "John",
      email: "john@example.com"
    }
  }
}

Iteration steps

Process collections:

action ProcessUsers {
  get "/users"

  // Iterate all items
  for user in response.users {
    // Process each user
  }

  // With filtering
  for user in response.users where .status == "active" {
    // Process only active users
  }
}

Transformation steps

Transform data shapes:

action TransformData {
  get "/users"

  for user in response.users {
    map user -> StandardUser {
      id: .id,
      fullName: concat(.firstName, " ", .lastName),
      email: lowercase(.email),
      createdAt: parseDate(.created_at)
    }

    store user -> users { key: .id }
  }
}

See the Vague documentation for expression syntax.

Validation steps

Check data constraints:

action ValidateData {
  get "/users"

  for user in response.users {
    validate user {
      assume .id is string,
      assume length(.name) > 0,
      assume .email contains "@",
      assume .age >= 18
    }

    store user -> users { key: .id }
  }
}

Store steps

Persist data:

action SaveData {
  get "/users"

  // Store entire response
  store response -> allData

  // Store with key
  store response.users -> users { key: .id }

  // Upsert mode
  store response.users -> users { key: .id, upsert: true }

  // Partial update
  store response.users -> users { key: .id, partial: true }
}

Pattern matching steps

Route data based on shape:

action HandleResponse {
  get "/users"

  match response {
    { error: e, code: 401 } -> jump RefreshToken then retry,
    { error: e, code: 429 } -> retry { maxAttempts: 5 },
    { error: e } -> abort "API error",
    { users: _ } -> continue,
    _ -> abort "Unexpected response"
  }
}

Nested actions

Actions can reference other actions via jump:

action Main {
  get "/data"

  match response {
    AuthError -> jump RefreshAuth then retry,
    _ -> continue
  }
}

action RefreshAuth {
  post "/auth/refresh" { body: { token: env("REFRESH_TOKEN") } }
  // Token is automatically used for subsequent requests
}

Action composition in pipelines

Actions are composed in the run statement:

mission DataPipeline {
  action Fetch { /* ... */ }
  action Transform { /* ... */ }
  action Export { /* ... */ }

  // Sequential
  run Fetch then Transform then Export

  // Parallel groups
  run [FetchA, FetchB] then Merge then Export
}

Variable scope

Variables are scoped to their action and nested contexts:

action ProcessData {
  get "/users"  // response is set

  for user in response.users {
    // user is available here
    // response is still available

    map user -> processed {
      // user and response available
    }
    // processed is available
  }

  // user is no longer available here
  // response is still available
}

Best practices

Single responsibility

Each action should do one thing well:

// Good: focused actions
action FetchUsers {
  get "/users"
  store response -> rawUsers
}

action TransformUsers {
  for user in rawUsers {
    map user -> StandardUser { /* ... */ }
    store user -> users { key: .id }
  }
}

// Avoid: doing too much
action DoEverything {
  get "/users"
  get "/orders"
  // transform both
  // export to multiple places
}

Handle errors at action boundaries

action FetchWithErrorHandling {
  get "/users"

  match response {
    ErrorResponse -> queue failures { item: response },
    _ -> store response -> users { key: .id }
  }
}

Use descriptive names

// Good
action FetchActiveCustomersWithOrders { }
action TransformToQuickBooksFormat { }
action ExportToDataWarehouse { }

// Avoid
action Step1 { }
action Process { }