Product Announcement: New Power Duo In Town

Feature Spotlight #1: Read-only Storage

What is it? 

Currently, big data tables can be hard to work with. The process of running a transformation or merely reading data in Storage involves an initial step called input mapping. This step can slow down your data operations, causing delays.

We've addressed this with our new feature, Read-only storage, which accelerates this process significantly. It enables users and applications to access stored data more quickly by bypassing the input mapping stage altogether.

How does it help you? 

Read-only storage unlocks many benefits:

• Faster setup time: The platform provides immediate access to tables in storage, removing the need to define input and output mapping manually.
• Reduced execution time: Transformations run faster without the backend’s overhead of mapping inputs and outputs before executing a script. This is particularly beneficial for large tables that take a considerable amount to load using standard input mapping before executing an actual transformation.
• Faster storage updates: Without read-only storage, updating data in storage with output mapping caused multiple data movement operations, resulting in slower updates.
• Simplified complex orchestrations: Read-only storage reduces the number of operations required when moving data from a source to a workspace. This simplification is especially useful when you want other apps to have easy access to stored data.

How does it work?

For new or existing configurations simply enable the option "Grant read-only access to all storage data". 

‍

Rest assured, users still maintain control. In situations where access needs to be restricted, (e.g. sharing a Workspace with external collaborators), users can select "Direct Storage Access" and manually define input and output mappings.

For more information, please refer to the user documentation.

How can I start using read-only storage?

Read-only storage is free of charge to all Keboola users.

Check the component's documentation to see if it supports native data types. Our database extractors that are maintained by Keboola already provide data types.

‍

Feature Spotlight #2: Native Data Types 

What is it?

Back in the day, Keboola stored data in Storage as text (VARCHAR NOT NULL) by default, and you had to manually correct or update the data type mappings before any extraction or loading took place.

Now, Keboola streamlines the process of data type propagation from the source to the storage, removing the need for manual definitions or updates.

How does it help you? 

Eliminating the need for manual data type adjustments provides many benefits:

• Simplified data manipulations: There's no longer a need to write casting statements in your code, or spend time manually adjusting the correct data types in storage.
• Faster loading and unloading processes: Read-Only Input Mapping (check feature above) seamlessly handles data types, reducing the need for casting operations and speeding up your loading and unloading processes.
• Improved data quality: Enhance the data accuracy and consistency throughout your pipelines without additional effort.

How does it work? 

The specific mechanism depends on your project's backend, but there are two ways that native data types are implemented:

1. Manual via API: You can use type definitions using the table-definition endpoint.
2. Automated via Components: DB Extractors and transformations will match the storage backend data types by default. If mapping from the source to storage isn't possible, the component will revert to base types.

For a more in-depth understanding, please refer to our user documentation on how to leverage native data types.

How can I start using the native data types?

The feature is free of charge for all Keboola users, automatically activated for new PAYG projects. For existing projects, kindly request activation through our support team.

‍

Try new features for free.

‍