Flow Builder - the collaborative new feature that speeds up data pipeline deployment
Keboola is proud to announce the launch of its new feature: the Flow builder.
The Flow Builder is a new feature that allows you to visually build end-to-end data pipelines from a single drag-and-drop user interface.
Building data pipelines can be extremely complicated - from juggling the challenges of multiple data sources and formats to requiring advanced programming knowledge for transformations and understanding the intricacies of how to change your loading process to tailor it to every data destination’s specific conditions.
But Keboola already simplified each step of the process with its reusable ready-made components. Simply select the right component to either extract, transform, or load your data.
Now Keboola takes it a step further by joining all the components into a single Flow Builder. You can create data flows by dragging and dropping the components together on a single screen, and even schedule and automate your data pipelines with a button click.
This simplicity brings you five main benefits.
The Flow Builder offers five benefits to Keboola’s users:
Let’s now build a new data flow to see how to unlock the benefits in practice.
We will build an end-to-end data flow using Flow Builder in 3 steps.
Log into your Keboola account to get started.
Don’t have a Keboola account yet? No worries! Keboola offers an always-free (no credit card required) tier that allows you to follow along. Create an account here.
Once in Keboola, select from the top menu Flows > Flows > New Flow:
Give your new flow a name and an easy to understand description. Then click create flow.
This will open up the Flow Builder view, where you can view and build your data flow.
Start by selecting the components to collect your data from your data sources. The general process is simple:
We will show you an end-to-end example of setting up ready-made components by walking you through reading data files from Google Drive.
We start by clicking on Select Components > Google Drive to collect a file from our Google Drive folder.
We proceed by clicking on the Google Drive component> show details cog > Select or Create Configuration to add new configuration. The configuration grants access to Keboola so it can read the data files on your Google Drive:
Name the configuration (e.g., “Google Drive Config”) and click on Create (shown below) and the Configure (not shown):
The wizard guides you through Google’s authorization to give Keboola access to our Google Drive (not shown here, but super easy to follow).
Continue adding other ready-made components and organize them in a logical flow or select the ones you want to run in parallel using drag & drop function.
For each flow:
Once you’ve built your flow end-to-end simply click on Run flow, to set the data pipeline into action.
Within the Flow Builder, click on Set schedule under Automate to set up the orchestration that will automatically run your job at your preferred recurring time.
That’s the power of automation for you - set it and automate it.
In the tab “All runs” you can check the run history of your flow with its detailed breakdown to each task.
Log into your Keboola account and start building beautiful visual flows today.
Don’t have a Keboola account yet? No worries!
Keboola offers an always-free (no credit card required) tier that allows you to follow along. Create an account here.