We’re excited to announce the Common Availability of Lakehouse Federation for Google BigQuery and the Public Preview for Oracle and Teradata. Now, you possibly can join, uncover, govern, and question information from these sources by means of Unity Catalog—with out migration or ETL. This makes information entry simpler whereas making certain an open, interoperable lakehouse structure.
Unify your information and governance throughout distributed platforms with the Lakehouse Federation
Lakehouse Federation permits customers to question and analyze information throughout disparate databases, information warehouses and catalogs with out duplicating or transferring information. By integrating exterior information sources immediately into the Unity Catalog, organizations can:
- Acquire a unified view of your total information property: Mechanically classify and uncover each structured and unstructured information in a single platform, empowering your group to securely entry and discover all accessible information—wherever it resides.
- Question and analyze information seamlessly with a single engine: Allow quick, advert hoc evaluation and prototyping throughout all of your information, analytics, and AI workloads with out the necessity for ingestion. A unified engine optimizes question efficiency with superior question planning, caching, and cross-source execution, permitting you to effectively mix information from a number of platforms in a single question.
- Guarantee constant information safety and governance: Apply a unified permission mannequin to implement entry guidelines throughout all information sources. Implement row- and column-level safety, tag-based insurance policies, and centralized auditing whereas sustaining full visibility into information utilization. Meet compliance necessities effortlessly with built-in information lineage and auditability.
Lakehouse Federation helps a various set of platforms, together with MySQL, PostgreSQL, Teradata, Oracle, Amazon Redshift, Salesforce Information Cloud, Snowflake, Microsoft SQL Server, Azure Synapse (SQL Information Warehouse), Google BigQuery, and Hive metastore, with extra connectors coming quickly.
Common Availability of Lakehouse Federation for BigQuery
With the GA launch of Lakehouse Federation for Google BigQuery, customers can now join BigQuery information to Databricks with out the necessity for information motion or duplication. This allows them to run workloads immediately on BigQuery information from their Databricks surroundings on GCP, Azure or AWS, whereas leveraging Unity Catalog’s fine-grained governance for safe and environment friendly information administration.
This integration permits for:
- Unified information administration: Seamlessly uncover and govern BigQuery information alongside different sources inside Unity Catalog, making certain constant entry management and governance.
- Optimized question efficiency: Make the most of BigQuery’s compute energy whereas benefiting from Databricks’ superior question optimization for quicker insights.
- Improved collaboration: Empower groups to work throughout cloud environments effectively with a unified interface for querying and analyzing information throughout a number of platforms.
Lakehouse Federation for BigQuery has seamlessly linked our Google Cloud and Azure environments. With out the necessity for information motion or duplication, we will now leverage Databricks on Azure to run Python and SQL workloads immediately on BigQuery information, all whereas benefiting from Unity Catalog’s fine-grained governance. It is a game-changer for our multicloud technique, streamlining operations and enhancing analytics
— Lisa Fiege, Information Science Engineer, Taylor Farms
The Google BigQuery connector is now typically accessible beginning with DBR model 16.1 and can quickly be accessible in Databricks SQL.
Public Preview for Teradata and Oracle Connectors
Along with the GA announcement for BigQuery, Databricks can also be introducing Teradata and Oracle connectors into Public Preview. These new connectors will permit customers to increase the attain of Lakehouse Federation to much more information sources, additional unifying the info estates.
Get Began
Learn our documentation to get began with Lakehouse Federation connectors for Google BigQuery, Oracle and Teradata: