“Reverse ETL” with Keboola

How To

May 6, 2021

Learn about the way Keboola operationalizes data

TL;DR: Yes, you can do it. And no, you don’t need a separate tool for it.

“Reverse ETL” is a fairly recent addition to the data engineer’s dictionary. While you can read articles upon articles about it (there’s a pretty good ‘primer’ in the Memory Leak blog), it can be summarized as being the art and science of taking data from your data warehouse and sending it somewhere other than BI - generally into other tools and systems where it becomes operational.

So, what does it mean in practice, why do we need a new term, and how does Keboola play a role in that space?

Reverse ETL in practice

When you treat your data as an asset, chances are there’s a lot of value in there. Analytics is not the only application. For example, you may model some smart customer segmentation and use the segments to enrich records in your data catalog. Having that information in a BI dashboard is great, but it’s even better when you have it synchronized in your CRM or e-mail marketing platform. Or, you can use contract information from your CRM to predict future invoicing. While doing that, why not make that data available to your accounting system to actually generate those invoices? You may need to clean and standardize mailing addresses for updating them in your e-commerce platform.

You get the gist - there’s literally an unlimited number of ideas on how you can use data to automate and improve business processes (read on for some specific examples). How often is there a dashboard presented with a completely predictable outcome/task for the user of the dashboard to perform? You may want to automate that task in the first place.

Why is it a separate category?

There are tools that specialize in this type of integrations. Although there really isn’t a need for it in our opinion (it’s just a data flow like any other!), it has become a differentiator in the marketplace. It’s just a reason to add another tool into what is (probably) already a complicated data stack. The thing that helps those companies is that traditional ETL tools generally struggle with writing into anything other than a SQL database or data warehouse of some kind.

So, if you do have the traditional stack (FiveTran, DBT, and Snowflake, for example) and you want to send data back to Salesforce or Mailchimp or Shopify, then you’re out of luck. Yes, you will need to add another vendor to your stack. And then probably another, because when you have four different tools, why not throw in a fifth just to consolidate user management?

How about Keboola?

“Reverse ETL” doesn’t excite us very much as a category on its own. It’s just a use case, a particular type of data flow - not something that requires a separate tool. Keboola offers 50+ data destinations and the list is always growing. Therefore, setting up a “Reverse ETL” flow in Keboola is no different than doing data acquisition in the first place - using the same building blocks, the same principles, staying within the secure perimeter of the platform, with no need for additional knowledge or vendor selection. In addition to the ever-growing list of data destinations that we support out of the box, we also offer a generic writer that enables low-code integration to send data into essentially any REST API.

“Reverse ETL” flows at Keboola

In the Keboola data-stack-as-a-service, we discuss “data acquisition” and “data consumption” use cases. They generally have a data catalog at the core, and either feed data into it or take it out for consumption by various systems. Our customers have literally been doing this for years. As an example, and perhaps for inspiration, here’s a diagram of Keboola (the company’s) own data environment and the connected systems. It goes without saying that everything is managed within the Keboola platform. The “Reverse ETL” flows are highlighted and numbered, with the descriptions below:

Academy badges into Webflow - when our Academy team assigns badges to Keboola Academy alumni, their achievements get updated in a collection in Webflow. This automatically generates their personal “diploma” pages.
New integration listings into Webflow - when a new component gets published which supports a new integration, it is automatically published in a Webflow collection that powers the Integrations page.
Webinar registrations into Zoom - as we collect webinar registrations via several channels, it makes sense for us to consolidate them first before submitting to Zoom.
New tickets to Zendesk - some events (such as changes to project settings) require manual intervention from our tech support team. Instead of a separate dashboard that they’d have to check, we simply auto-create a ticket in Zendesk, allowing the task to fit seamlessly into their workflow.
Marketing spreadsheet output - this flow serves various needs, such as listing webinar registrations, reporting on campaign spend, etc.
Integration lists for partners - we also use GoogleSheets to share updated lists of integrations that Keboola supports out of the box with our partners.
Update of mailing lists and trigger events into Customer.io - we process users’ preferences and statuses in Keboola and create lists in customer.io accordingly. We also provide the platform with multiple “trigger events”, which allow us to send adaptive email campaigns. This workflow also handles user notifications, such as credit spend overview, low credit warnings, etc. (see this blog post for more on the topic).
Known user profile updates to Lytics for website personalization - this is an ongoing project, but we are working on personalization of our website for known users. This flow keeps the Lytics platform informed of their preferences.
Various notifications to Slack - examples include activity on the Academy for its team, new leads, project expirations, and more.
New leads, contacts, and events into Salesforce from various sources - instead of point-to-point integrations, we collect all new leads from web forms, registrations, sign-ups to our various assets, etc. in Keboola Connection first. We then pass only processed and properly tagged information into Salesforce. At the same time, we recognize email addresses belonging to existing accounts and assign them automatically, notifying the account owners (more on writing to SFDC here).
Financial information into Google Sheets - accountants. They just want their spreadsheets; you can’t make them happy otherwise, regardless of how good your data environment is! This flow keeps their models alive with fresh data.
Live presentation decks in Google Slides - we also use the GoogleSheets writer to update repeatedly-used presentation decks for events like all-hands or board meetings (more on that use case here).
Telemetry data sharing with customers - several Keboola customers receive customized telemetry data feeds via their Keboola projects, which they then consume in various ways.
Generated invoices into the accounting systems Xero and FlexiBee - while Salesforce handles the prescription of recurring payments, we handle the preparation of the data for our various accounting systems of our respective entities. We produce files that can be easily imported to make our bookkeepers’ lives easier (more in an older blog post here).

Visit our integrations page to see all of the currently available data destinations. Alternatively, just sign up for a free Keboola account and give it a spin!

#getsmarter

Oops! Something went wrong while submitting the form.