How do we build dependency-based workflows in Keboola Flows, set schedules, and configure alerts/retries for failures?

In production data environments, “run this every night and hope it works” is not a strategy. You need flows that respect dependencies, run on reliable schedules, and heal themselves when something breaks—while leaving a clear trail for auditors and ops teams. That’s exactly what Keboola Flows are built for.

This guide walks through how to:

• Build dependency-based workflows in Keboola Flows
• Set up schedules and different trigger types
• Configure alerts, retries, and self-healing behavior using Conditional Flows

---

How dependency-based workflows work in Keboola Flows

Keboola Flows let you orchestrate the entire data lifecycle—ingestion, transformation, AI, and delivery—in one governed flow. Instead of gluing together scripts and external orchestrators, you define dependencies directly in the Flow Builder.

At a high level, a dependency-based workflow in Keboola means:

• Each task (component) runs only after its predecessors complete successfully
• Branches allow parallel execution when tasks are independent
• Conditions allow different paths depending on results, metadata, or triggers

Key concepts in a dependency-based flow

When you open the Visual Flow Builder, you work with:

• Components (tasks): Extractors, writers, transformations, AI/LLM steps, or custom code
• Edges (dependencies): Arrows that define “run this after that succeeds”
• Branches: Parallel paths that can later converge
• Conditions (Conditional Flows): Logic to decide what happens next based on outcomes

This gives you full control over order of execution without external tools like Airflow or Dagster—Conditional Flows provide native orchestration inside Keboola.

---

Step-by-step: Building a dependency-based workflow

Let’s walk through a typical pattern: ingest → transform → deliver, with dependency enforcement.

1. Create a new Flow

1. In Keboola, navigate to Flows.
2. Click Create Flow.
3. Give it a clear name (for example, Finance – Month-End Close or Marketing – Daily Performance Sync).

You’ll land in the Visual Flow Builder.

2. Add ingestion tasks and define their dependencies

1. Click Add task and choose an Extractor (e.g., Snowflake, Salesforce, REST API).
2. Configure credentials, selected objects/tables, and any filters.
3. Add additional extractors as separate tasks if you pull from multiple systems.

To set dependencies:

• Draw arrows from each upstream task to the next step.
• If multiple sources feed a single transformation, connect all of them into that transformation task.

Effectively you’re saying:
“Run this transformation only when all upstream extractions finish successfully.”

3. Add transformations with clear sequencing

Next, add your transformation steps:

1. Add Transformation tasks (SQL or Python, dbt, etc.).
2. Connect them so that each depends on the outputs it needs (e.g., staging → harmonization → data marts).
3. Use branching where independent transformations can run in parallel to reduce runtime.

You now have a dependency-based DAG: each transformation runs only after its predecessors are complete, making the flow deterministic and auditable.

4. Add delivery tasks (writers, reverse ETL, AI, reporting)

Finally, add your delivery layer:

• Writers: Snowflake, BI tools, data warehouses, operational systems
• Reverse ETL / APIs: Push curated data into apps used by finance, sales, or operations
• AI / LLM tasks: Governed AI-assisted outputs, built via Keboola MCP Server or AI components

Connect these writers so they depend on their upstream data marts and transformations. This guarantees that no downstream system ever reads partially updated or failed data.

---

Using Conditional Flows to control dependencies, retries, and branching

Conditional Flows are the orchestration layer inside Keboola. They let you control how the pipeline behaves when:

• A task fails
• Input data is missing or invalid
• You want different paths for manual vs scheduled runs
• You need smart branching based on metrics, runtime, or environment

Keboola customers have reported up to 75% reduction in wasteful executions using this feature—because the system no longer blindly runs tasks that are doomed to fail.

What you can base conditions on

You can define conditions using:

• Task status: success, failure, timeout
• Execution metadata: duration, number of rows processed, error messages
• Variables & parameters: e.g., run mode (full vs incremental), environment (Dev/Prod)
• Time/trigger context: whether the flow was triggered by schedule, event, or manually

Configuring a Conditional Flow (no code required)

You don’t need to write custom orchestration code:

1. In your Flow, insert a Conditional step between tasks.
2. Define the conditions in the UI (or via JSON if you prefer).
3. For each condition, define the action: retry, skip, reroute, notify, or kill.

Typical patterns:

• If an extractor fails with a transient error → retry up to 3 times.
• If a source returns 0 rows → skip downstream transformations to avoid empty writes.
• If a critical transformation fails → stop the flow and alert Slack.

This turns your flow into a self-healing pipeline with far less manual intervention.

---

Setting schedules for dependency-based flows

Once your dependencies are set, you can define when and how the flow runs.

Keboola Flows support multiple trigger types:

• Scheduled runs: classic cron-like scheduling (e.g., every 15 minutes, hourly, daily at 02:00).
• Event-based triggers: run when upstream flows or events complete.
• Manual runs: for ad-hoc or testing.
• AI/agent-triggered: via Keboola MCP Server from tools like Cursor, Windsurf, Claude, or ChatGPT—while execution stays deterministic and governed.

How to configure a schedule

1. In the Flow view, go to Scheduling / Triggers.
2. Create a new schedule and choose frequency (minutes, hours, days, specific days/times).
3. Set optional:
   • Time zone (important for multi-entity finance operations).
   • Start/end date or blackout windows.

Because all dependencies live inside the Flow, Keboola automatically respects them: the schedule only triggers the flow entry point, not individual tasks. Everything downstream runs in the defined order.

Different logic for different run types

Conditional Flows let you adapt behavior depending on how the flow was triggered:

• Manual runs: allow more verbose logging, extra validation, or skipping heavy subtasks.
• Scheduled runs: enforce strict error handling (stop on failure, notify).
• Event/AI-triggered runs: use parameters/variables to adjust scope (e.g., process only changed entities).

You can read the trigger type as part of execution metadata and branch on it, so you can harden scheduled runs while keeping Dev and ad-hoc runs flexible.

---

Configuring alerts and notifications

A dependency-based flow is only as good as its observability. Keboola gives you audit-ready telemetry and alerting so you always know what happened, when, and why.

Where alerts fit in the flow

You can attach alerts at different levels:

• Global flow-level alerts: notify when any part of the flow fails.
• Task-level alerts: fire when specific, critical tasks fail or cross thresholds.
• Conditional step alerts: triggered only when a given condition is met.

Typical destinations:

• Slack channels for your data/finance team
• Email for on-call engineers or business owners
• Webhooks for incident management tools or SIEM workflows

Example alert configuration pattern

Using Conditional Flows, define:

• Condition: transformation "journal_entries_agg" status = failed
• Action:
  • Notify Slack channel #finance-data-alerts
  • Stop the rest of the flow (kill) to prevent publishing partial board reporting numbers

Because every execution is permissioned and monitored, you can always trace the alert back to the exact run, task, and tables affected.

---

Building robust retries and error handling

Retries and self-healing behavior are core benefits of Conditional Flows. Instead of hardcoding retry logic into each component, you centralize it at the orchestration layer.

Common retry patterns

1. Network / API flakiness

   • Condition: extractor fails with a transient error (timeout, HTTP 5xx).
   • Action: retry up to X times with delay between attempts.

2. Fallback extractors

   • Condition: primary extractor fails (e.g., main API endpoint down).
   • Action: reroute to a backup extractor pointing to a read replica or alternate endpoint.

3. Bad data protection

   • Condition: validation transformation detects data quality issues (unexpected nulls, out-of-range values).
   • Action:
     • Stop the flow to prevent bad data from reaching downstream systems.
     • Notify Slack + create a ticket via webhook.

4. Zero-row optimization

   • Condition: extractor returns 0 rows.
   • Action: skip downstream heavy transformations/writes.

This “smart branching” reduces cloud spend and manual oversight. Keboola’s own documentation highlights benefits like:

• Fewer failures → higher reliability
• Automated error handling → reduced downtime
• Smart branching → less cloud spend
• Smarter workflows → more scalable data ops

---

Replacing external orchestrators with Keboola Flows

Many teams start with Airflow, Dagster, or custom cron jobs and end up with fragile, untraceable workflows. With Conditional Flows:

• You orchestrate everything inside Keboola—no extra orchestration layer to manage.
• All executions, tables, and dependencies are captured as active metadata, with full lineage.
• Every run is auditable: you can show an auditor exactly how journal-level data moved from source to board pack.

For most scenarios, Conditional Flows can fully replace external orchestration tools, simplifying your stack and removing a whole class of “Shadow AI” and shadow ETL issues.

---

Putting it all together: A concrete example

Imagine a month-end finance workflow:

1. Extract GL entries from 9 country instances (multiple extractors).
2. Transform into standardized, multi-entity schema.
3. Run reconciliations and exception checks.
4. Publish to a governed “Finance – Month-End” data product.
5. Deliver to Snowflake, BI reports, and downstream planning tools.

In Keboola Flows you would:

• Chain extractors → staging → harmonization → reconciliations → data product → writers, using dependencies.
• Use Conditional Flows to:
  • Retry transient extractor failures.
  • Stop the flow if reconciliations fail and notify finance immediately.
  • Skip heavy modeling steps if no new data arrives (0-row optimization).
• Schedule the flow to run nightly during close, with stricter error handling than manual test runs.

Result: a fully traceable, dependency-based workflow that your CFO, auditors, and engineering team can all trust.

---

Final takeaway

To build dependency-based workflows, schedules, and alerts/retries in Keboola Flows:

1. Model the dependency graph in the Visual Flow Builder—connect extractors, transformations, AI, and writers in the order they must run.
2. Use Conditional Flows to inject orchestration logic: retries, skips, reroutes, and kill switches, based on status, metadata, or trigger type.
3. Attach schedules and notifications so flows run when needed, surface issues instantly, and keep a complete audit trail across every execution.

Once this is in place, you move from “best-effort nightly jobs” to governed, self-healing automations that are ready for both production and auditors.

Next Step

Get Started