Create an automation

• 1: Create a Slack automation
• 2: Create a webhook automation

1 - Create a Slack automation

This page shows how to create a Slack automation. To create a webhook automation, refer to Create a webhook automation instead.

At a high level, to create a Slack automation, you take these steps:

1. Add a Slack integration, which authorizes W&B to post to the Slack instance and channel.
2. Create the automation, which defines the event to watch for and the channel to notify.

Add a Slack integration

A team admin can add a Slack integration to the team.

1. Log in to W&B and go to Team Settings.

2. In the Slack channel integrations section, click Connect Slack to add a new Slack instance. To add a channel for an existing Slack instance, click New integration.

   

3. If necessary, sign in to Slack in your browser. When prompted, grant W&B permission to post to the Slack channel you select. Read the page, then click Search for a channel and begin typing the channel name. Select the channel from the list, then click Allow.

4. In Slack, go to the channel you selected. If you see a post like [Your Slack handle] added an integration to this channel: Weights & Biases, the integration is configured correctly.

Now you can create an automation that notifies the Slack channel you configured.

View and manage Slack integrations

A team admin can view and manage the team’s Slack instances and channels.

1. Log in to W&B and go to Team Settings.
2. View each Slack destination in the Slack channel integrations section.
3. Delete a destination by clicking its trash icon.

Create an automation

After you add a Slack integration, select Registry or Project, then follow these steps to create an automation that notifies the Slack channel.

• Registry
• Project

Create automations programmatically

You can also create Slack automations using the W&B API. This enables you to automate the creation and management of automations as part of your MLOps workflows.

Prerequisites

Before creating automations programmatically:

1. Ensure you have a Slack integration configured in your team settings
2. Install the W&B SDK: pip install wandb
3. Authenticate with your W&B API key

Note: Slack integrations must be created through the W&B UI due to the OAuth authentication flow required by Slack. Once configured, you can use the API to list available integrations and create automations that use them.

Checking for Slack Integrations

Before creating automations, verify that you have Slack integrations configured:

import wandb

# Initialize the API
api = wandb.Api()

# List all Slack integrations for your team
slack_integrations = list(api.slack_integrations(entity="your-team"))

if not slack_integrations:
    print("❌ No Slack integrations found!")
    print("Please configure a Slack integration in your team settings:")
    print(f"https://wandb.ai/{your-team}/settings/integrations")
else:
    print(f"✅ Found {len(slack_integrations)} Slack integration(s):")
    for integration in slack_integrations:
        print(f"   - Channel: {integration.channel_name}")
        print(f"     ID: {integration.id}")
        print(f"     Team: {integration.slack_team_name}")

Helper Function for Integration Selection

def get_slack_integration(entity, channel_pattern=None):
    """Get a Slack integration, optionally filtered by channel name pattern"""
    integrations = list(api.slack_integrations(entity=entity))
    
    if not integrations:
        raise ValueError(f"No Slack integrations found for {entity}. "
                        "Please configure one in team settings.")
    
    if channel_pattern:
        # Filter by channel name pattern
        matching = [i for i in integrations if channel_pattern in i.channel_name]
        if matching:
            return matching[0]
    
    # Return first integration if no pattern or no matches
    return integrations[0]

# Example usage
try:
    # Get integration for alerts channel
    alerts_integration = get_slack_integration("my-team", "alerts")
    print(f"Using Slack channel: {alerts_integration.channel_name}")
except ValueError as e:
    print(e)

• Registry
• Project

import wandb
from wandb.automations import OnAddArtifactAlias, SendNotification

# Initialize the W&B API
api = wandb.Api()

# Get the registry and collection
registry = api.registry("my-registry", entity="my-team")
collection = registry.collection("my-model-collection")

# Get the Slack integration
slack_integration = next(api.slack_integrations(entity="my-team"))

# Define the event: Trigger when "production" alias is added
event = OnAddArtifactAlias(
    scope=collection,
    filter={"alias": "production"}
)

# Define the action: Send notification to Slack
action = SendNotification.from_integration(
    slack_integration,
    title="Production Model Updated",
    text="A new model version has been promoted to production.",
    level="INFO"
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="production-model-alert",
    description="Notify team when a model is promoted to production"
)

print(f"Created automation: {automation.name}")

import wandb
from wandb.automations import OnCreateArtifact, SendNotification

# Initialize the W&B API
api = wandb.Api()

# Get the registry
registry = api.registry("my-registry", entity="my-team")

# Get the Slack integration  
slack_integration = next(api.slack_integrations(entity="my-team"))

# Define event at registry scope (applies to all collections)
event = OnCreateArtifact(
    scope=registry,
    filter={"artifact_type": "model"}
)

# Define the action
action = SendNotification.from_integration(
    slack_integration,
    title="New Model Registered",
    text="A new model artifact has been added to the registry."
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="registry-wide-model-monitor",
    description="Monitor all model artifacts across the registry"
)

import wandb
from wandb.automations import OnRunMetric, RunEvent, SendNotification

# Initialize the W&B API
api = wandb.Api()

# Get the project
project = api.project("my-ml-project", entity="my-team")

# Get the Slack integration
slack_integration = next(api.slack_integrations(entity="my-team"))

# Define the event: Trigger when accuracy drops below 0.85
event = OnRunMetric(
    scope=project,
    filter=RunEvent.metric("accuracy") < 0.85
)

# Define the action: Send alert to Slack
action = SendNotification.from_integration(
    slack_integration,
    title="⚠️ Low Accuracy Alert",
    text="Model accuracy has dropped below 85%",
    level="WARN"
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="low-accuracy-alert",
    description="Alert when model accuracy drops below threshold"
)

print(f"Created automation: {automation.name}")

import wandb
from wandb.automations import OnRunStateChange, RunState, SendNotification

# Initialize the W&B API
api = wandb.Api()

# Get the project
project = api.project("long-running-experiments", entity="my-team")

# Get the Slack integration
slack_integration = next(api.slack_integrations(entity="my-team"))

# Define the event: Trigger when a run finishes
event = OnRunStateChange(
    scope=project,
    to_state=RunState.finished
)

# Define the action
action = SendNotification.from_integration(
    slack_integration,
    title="✅ Training Complete",
    text="Your experiment has finished running.",
    level="INFO"
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="training-completion-notifier",
    description="Notify when training runs complete"
)

Managing automations via API

You can also list, update, and delete automations programmatically:

# List all automations for a project
automations = project.automations()
for automation in automations:
    print(f"- {automation.name}: {automation.description}")

# Update an automation (enable/disable)
automation.enabled = False
automation.save()

# Delete an automation
automation.delete()

Advanced patterns

Using multiple filters

Combine multiple conditions in your automation triggers:

from wandb.automations import OnRunMetric, RunEvent, SendNotification

# Trigger when accuracy is high AND loss is low
event = OnRunMetric(
    scope=project,
    filter=(RunEvent.metric("accuracy") > 0.95) & 
           (RunEvent.metric("loss") < 0.1)
)

Dynamic Slack channel selection

Route notifications to different channels based on your team structure:

# Find specific Slack integration by channel name
slack_integrations = api.slack_integrations(entity="my-team")
dev_channel = next(
    (ig for ig in slack_integrations if ig.channel_name == "ml-dev-alerts"),
    None
)

if dev_channel:
    action = SendNotification.from_integration(dev_channel)

View and manage automations

• Registry
• Project

2 - Create a webhook automation

This page shows how to create a webhook automation. To create a Slack automation, refer to Create a Slack automation instead.

At a high level, to create a webhook automation, you take these steps:

1. If necessary, create a W&B secret for each sensitive string required by the automation, such as an access token, password, or SSH key. Secrets are defined in your Team Settings.
2. Create a webhook to define the endpoint and authorization details and grant the integration access to any secrets it needs.
3. Create the automation to define the event to watch for and the payload W&B will send. Grant the automation access to any secrets it needs for the payload.

Create a webhook

A team admin can add a webhook for the team.

1. Log in to W&B and go to Team Settings page.

2. In the Webhooks section, click New webhook.

3. Provide a name for the webhook.

4. Provide the endpoint URL for the webhook.

5. If the webhook requires a Bearer token, set Access token to the secret that contains it. When using the webhook automation, W&B sets the Authorization: Bearer HTTP header to the access token, and you can access the token in the ${ACCESS_TOKEN} payload variable. Learn more about the structure of the POST request W&B sends to the webhook service in Troubleshoot your webhook.

6. If the webhook requires a password or other sensitive string in its payload, set Secret to the secret that contains it. When you configure the automation that uses the webhook, you can access the secret as a payload variable by prefixing its name with $.

   If the webhook’s access token is stored in a secret, you must also complete the next step to specify the secret as the access token.

7. To verify that the W&B can connect and authenticate to the endpoint:

   1. Optionally, provide a payload to test. To refer to a secret the webhook has access to in the payload, prefix its name with $. This payload is only used for testing and is not saved. You configure an automation’s payload when you create the automation. See Troubleshoot your webhook to view where the secret and access token are specified in the POST request.
   2. Click Test. W&B attempts to connect to the webhook’s endpoint using the credentials you configured. If you provided a payload, W&B sends it.

   If the test does not succeed, verify the webhook’s configuration and try again. If necessary, refer to Troubleshoot your webhook.

Now you can create an automation that uses the webhook.

Create an automation

After you configure a webhook, select Registry or Project, then follow these steps to create an automation that triggers the webhook.

• Registry
• Project

Create automations programmatically

You can also create webhook automations using the W&B API. This enables you to automate the creation and management of automations as part of your MLOps workflows.

Prerequisites

Before creating automations programmatically:

1. Ensure you have a webhook configured in your team settings
2. Install the W&B SDK: pip install wandb
3. Authenticate with your W&B API key
4. If your webhook requires authentication, ensure you have created the necessary secrets

Note: Webhooks and secrets must be created through the W&B UI for security reasons. The UI allows secure configuration of endpoints, authentication tokens, and secret management. Once configured, you can use the API to list available webhooks and create automations that use them.

Checking for Webhook Integrations

Before creating automations, verify that you have webhook integrations configured:

import wandb

# Initialize the API
api = wandb.Api()

# List all webhook integrations for your team
webhook_integrations = list(api.webhook_integrations(entity="your-team"))

if not webhook_integrations:
    print("❌ No webhook integrations found!")
    print("Please configure a webhook in your team settings:")
    print(f"https://wandb.ai/{your-team}/settings/webhooks")
else:
    print(f"✅ Found {len(webhook_integrations)} webhook integration(s):")
    for webhook in webhook_integrations:
        print(f"   - Name: {webhook.name}")
        print(f"     URL: {webhook.url_endpoint}")
        print(f"     ID: {webhook.id}")
        if hasattr(webhook, 'has_access_token') and webhook.has_access_token:
            print(f"     Auth: Has access token")

Helper Functions for Webhook Management

def get_webhook_integration(entity, name_pattern=None, url_pattern=None):
    """Get a webhook integration by name or URL pattern"""
    webhooks = list(api.webhook_integrations(entity=entity))
    
    if not webhooks:
        raise ValueError(f"No webhook integrations found for {entity}. "
                        "Please configure one in team settings.")
    
    # Filter by name pattern
    if name_pattern:
        matching = [w for w in webhooks if name_pattern in w.name]
        if matching:
            return matching[0]
    
    # Filter by URL pattern
    if url_pattern:
        matching = [w for w in webhooks if url_pattern in w.url_endpoint]
        if matching:
            return matching[0]
    
    # Return first webhook if no pattern or no matches
    return webhooks[0]

def list_webhook_requirements(webhook):
    """Display information about webhook configuration"""
    print(f"Webhook: {webhook.name}")
    print(f"  Endpoint: {webhook.url_endpoint}")
    
    if hasattr(webhook, 'has_access_token') and webhook.has_access_token:
        print("  ✓ Has access token configured")
        print("    Use ${ACCESS_TOKEN} in payload to access it")
    
    if hasattr(webhook, 'secrets') and webhook.secrets:
        print(f"  ✓ Has {len(webhook.secrets)} secret(s) configured")
        for secret in webhook.secrets:
            print(f"    Use ${{{secret}}} in payload to access secret '{secret}'")

# Example usage
try:
    # Get webhook for GitHub integration
    github_webhook = get_webhook_integration("my-team", url_pattern="github.com")
    list_webhook_requirements(github_webhook)
except ValueError as e:
    print(e)

Understanding Webhook Configuration

When creating webhook automations, it’s important to understand what was configured in the UI:

1. Endpoint URL: The URL where W&B will send POST requests
2. Access Token: Optional Bearer token for authentication (accessed via ${ACCESS_TOKEN})
3. Secrets: Additional secrets for the payload (accessed via ${SECRET_NAME})

Example of checking webhook capabilities:

# Check if webhook supports authentication
webhook = get_webhook_integration("my-team", "deployment-webhook")

# Build payload based on webhook configuration
payload = {
    "event": "model_ready",
    "model": "${artifact_version_string}",
}

# Add authentication if configured
if hasattr(webhook, 'has_access_token') and webhook.has_access_token:
    payload["auth"] = "${ACCESS_TOKEN}"  # W&B will substitute the actual token

# Create automation with appropriate payload
action = SendWebhook.from_integration(webhook, payload=payload)

• Registry
• Project

import wandb
from wandb.automations import OnAddArtifactAlias, SendWebhook

# Initialize the W&B API
api = wandb.Api()

# Get the registry and collection
registry = api.registry("model-registry", entity="my-team")
collection = registry.collection("llm-models")

# Get the webhook integration
webhook_integration = next(api.webhook_integrations(entity="my-team"))

# Define the event: Trigger when "production" alias is added
event = OnAddArtifactAlias(
    scope=collection,
    filter={"alias": "production"}
)

# Define the webhook payload with template variables
payload = {
    "event_type": "DEPLOY_MODEL",
    "model_info": {
        "artifact_version": "${artifact_version_string}",
        "collection": "${artifact_collection_name}",
        "entity": "${entity_name}",
        "project": "${project_name}",
        "author": "${event_author}",
        "event_type": "${event_type}"
    },
    "deployment_config": {
        "environment": "production",
        "auto_scale": True,
        "min_replicas": 2
    }
}

# Define the action: Send webhook with payload
action = SendWebhook.from_integration(
    webhook_integration,
    payload=payload
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="production-deployment-trigger",
    description="Trigger deployment pipeline when model is promoted to production"
)

print(f"Created automation: {automation.name}")

import wandb
from wandb.automations import OnCreateArtifact, SendWebhook

# Initialize the W&B API
api = wandb.Api()

# Get the registry
registry = api.registry("model-registry", entity="my-team")

# Get the webhook integration
webhook_integration = next(
    (w for w in api.webhook_integrations(entity="my-team")
     if "tracking-system" in w.url_endpoint)
)

# Define event at registry scope
event = OnCreateArtifact(
    scope=registry,
    filter={"artifact_type": "model"}
)

# Define payload with artifact metadata
payload = {
    "action": "register_model",
    "artifact": {
        "id": "${artifact_version}",
        "name": "${artifact_collection_name}",
        "metadata": "${artifact_metadata.model_type}",  # Access specific metadata key
        "created_by": "${event_author}"
    },
    "source": "wandb_registry"
}

# Define the action
action = SendWebhook.from_integration(
    webhook_integration,
    payload=payload
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="external-tracking-sync",
    description="Sync new models to external tracking system"
)

import wandb
from wandb.automations import OnRunMetric, RunEvent, SendWebhook

# Initialize the W&B API
api = wandb.Api()

# Get the project
project = api.project("model-training", entity="my-team")

# Get the webhook integration (GitHub repository dispatch)
github_webhook = next(
    (w for w in api.webhook_integrations(entity="my-team")
     if "api.github.com" in w.url_endpoint)
)

# Define the event: Trigger when model performance is good
event = OnRunMetric(
    scope=project,
    filter=(RunEvent.metric("val_accuracy") > 0.95) & 
           (RunEvent.metric("val_loss") < 0.05)
)

# Define GitHub repository dispatch payload
payload = {
    "event_type": "MODEL_READY_FOR_REVIEW",
    "client_payload": {
        "artifact_version": "${artifact_version_string}",
        "collection": "${artifact_collection_name}",
        "project": "${project_name}",
        "metrics": {
            "event_type": "${event_type}",
            "author": "${event_author}"
        },
        "author": "${event_author}",
        "entity": "${entity_name}"
    }
}

# Define the action
action = SendWebhook.from_integration(
    github_webhook,
    payload=payload
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="high-performance-model-review",
    description="Trigger review workflow for high-performing models"
)

print(f"Created automation: {automation.name}")

import wandb
from wandb.automations import OnRunMetric, RunEvent, SendWebhook

# Initialize the W&B API
api = wandb.Api()

# Get the project
project = api.project("research-experiments", entity="my-team")

# Get a specific webhook by name or URL
webhook_integration = next(
    w for w in api.webhook_integrations(entity="my-team")
    if w.name == "experiment-tracker"
)

# Define the event: Trigger when a run completes with good metrics
event = OnRunMetric(
    scope=project,
    filter=RunEvent.metric("final_loss") < 0.1
)

# Define custom payload with run summary
payload = {
    "experiment": {
        "event_type": "${event_type}",
        "project": "${project_name}",
        "entity": "${entity_name}",
        "author": "${event_author}",
        "artifact_version": "${artifact_version_string}"
    },
    "notification": {
        "type": "experiment_complete",
        "priority": "normal",
        "recipient": "${event_author}"
    }
}

# Define the action
action = SendWebhook.from_integration(
    webhook_integration,
    payload=payload
)

# Create the automation
automation = api.create_automation(
    event >> action,
    name="experiment-completion-tracker",
    description="Send notification when experiments achieve low loss"
)

Managing webhook automations via API

You can list, update, and manage webhook automations programmatically:

# List all automations for a project or registry
automations = project.automations()  # or registry.automations()

# Filter automations by type
webhook_automations = [
    a for a in automations 
    if a.action.action_type == "GENERIC_WEBHOOK"
]

# Update webhook payload
for automation in webhook_automations:
    if automation.name == "my-webhook-automation":
        # Modify the automation configuration
        automation.action.request_payload = updated_payload
        automation.save()

# Delete an automation
automation.delete()

Advanced webhook patterns

Using secrets in webhooks

Access secrets in your webhook payloads for secure authentication:

# Assuming you have a secret named "DEPLOY_TOKEN" configured
payload = {
    "auth_token": "${DEPLOY_TOKEN}",  # Secret accessed with $ prefix
    "deployment": {
        "model": "${artifact_version_string}",
        "environment": "production"
    }
}

Conditional webhook routing

Route webhooks to different endpoints based on conditions:

# Get multiple webhook integrations
staging_webhook = next(
    w for w in api.webhook_integrations(entity="my-team")
    if "staging" in w.name
)
production_webhook = next(
    w for w in api.webhook_integrations(entity="my-team")
    if "production" in w.name
)

# Create different automations for different conditions
# Staging automation for experiments
staging_event = OnRunMetric(
    scope=project,
    filter=RunEvent.metric("val_accuracy") > 0.90
)
staging_automation = api.create_automation(
    staging_event >> SendWebhook.from_integration(staging_webhook),
    name="staging-deployment"
)

# Production automation for higher threshold
prod_event = OnRunMetric(
    scope=project,
    filter=RunEvent.metric("val_accuracy") > 0.95
)
prod_automation = api.create_automation(
    prod_event >> SendWebhook.from_integration(production_webhook),
    name="production-deployment"
)

Error handling and validation

Add validation when creating webhook automations:

try:
    # Verify webhook integration exists
    webhook_integrations = list(api.webhook_integrations(entity="my-team"))
    if not webhook_integrations:
        raise ValueError("No webhook integrations found. Please configure webhooks first.")
    
    # Create automation with error handling
    automation = api.create_automation(
        event >> action,
        name="validated-webhook-automation",
        description="Webhook automation with validation"
    )
    print(f"Successfully created: {automation.name}")
    
except Exception as e:
    print(f"Failed to create automation: {e}")

View and manage automations

• Registry
• Project

Payload reference

Use these sections to construct your webhoook’s payload. For details about testing your webhook and its payload, refer to Troubleshoot your webhook.

Payload variables

This section describes the variables you can use to construct your webhook’s payload.

Payload examples

This section includes examples of webhook payloads for some common use cases. The examples demonstrate how to use payload variables.

• GitHub repository dispatch
• Microsoft Teams notification
• Slack notifications

on:
repository_dispatch:
  types: BUILD_AND_DEPLOY

{
  "event_type": "BUILD_AND_DEPLOY",
  "client_payload": 
  {
    "event_author": "${event_author}",
    "artifact_version": "${artifact_version}",
    "artifact_version_string": "${artifact_version_string}",
    "artifact_collection_name": "${artifact_collection_name}",
    "project_name": "${project_name}",
    "entity_name": "${entity_name}"
    }
}

${event_type} --> "LINK_ARTIFACT" or "ADD_ARTIFACT_ALIAS"
${event_author} --> "<wandb-user>"
${artifact_version} --> "wandb-artifact://_id/QXJ0aWZhY3Q6NTE3ODg5ODg3""
${artifact_version_string} --> "<entity>/model-registry/<registered_model_name>:<alias>"
${artifact_collection_name} --> "<registered_model_name>"
${project_name} --> "model-registry"
${entity_name} --> "<entity>"

{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"summary": "New Notification",
"sections": [
  {
    "activityTitle": "Notification from WANDB",
    "text": "This is an example message sent via Teams webhook.",
    "facts": [
      {
        "name": "Author",
        "value": "${event_author}"
      },
      {
        "name": "Event Type",
        "value": "${event_type}"
      }
    ],
    "markdown": true
  }
]
}

{
    "text": "New alert from WANDB!",
"blocks": [
    {
            "type": "section",
        "text": {
            "type": "mrkdwn",
            "text": "Registry event: ${event_type}"
        }
    },
        {
            "type":"section",
            "text": {
            "type": "mrkdwn",
            "text": "New version: ${artifact_version_string}"
        }
        },
        {
        "type": "divider"
    },
        {
            "type": "section",
        "text": {
            "type": "mrkdwn",
            "text": "Author: ${event_author}"
        }
        }
    ]
}

Troubleshoot your webhook

Interactively troubleshoot your webhook with the W&B App UI or programmatically with a Bash script. You can troubleshoot a webhook when you create a new webhook or edit an existing webhook.

For details about the format W&B uses for the POST request, refer to the Bash script tab.

• W&B App UI
• Bash script