This can be a collaborative submit from Databricks and dbt. We thank Florian Eiden, Workers Product Supervisor at dbt, for his contributions.
dbt is an information transformation framework that permits information groups to collaboratively mannequin, check and doc information in information warehouses. It’s fashionable with Databricks clients, who use it to rework and put together information within the lakehouse. Many of those clients together with Conde Nast, Pink Ventures, Loft and Aktify additionally use dbt Cloud to develop, check, schedule, and examine dbt information fashions via a collaborative web-based UI.
As we speak, we’re pleased to announce that dbt Cloud has added help for dbt-databricks, a purpose-built dbt adapter for Databricks. This adapter lets dbt Cloud customers benefit from Unity Catalog’s three-level namespace, enabling clear separation of tables and views managed by dbt fashions in addition to information governance utilizing customary SQL. The adapter additionally consists of nice out-of-the-box efficiency and sensible defaults, which implies much less tuning and toil for information groups.
All new Databricks initiatives created in dbt Cloud will routinely use the brand new dbt-databricks adapter. Current initiatives must be migrated. Now we have printed a useful migration information so you’ll be able to safely check and migrate your venture.
We’re excited in regards to the partnership between Databricks and dbt Cloud. We’d love to listen to your suggestions on Databricks Neighborhood or within the #db-databricks-and-spark dbt neighborhood Slack channel.
