Setup dbt Core on your machine in 5 minutes or less!

dbt is a data transformation tool that enables data teams to implement robust analytics engineering practices to improve the quality of their data. dbt has been a pioneer in a number of areas around data transformation, and as such is widely adopted by the industry today. You can collaborate on data models, version them, and test and document your queries before safely deploying them to production. dbt comes with 2 versions - one is a cloud version, and the other is dbt-core, an open source version of dbt that everyone can get started with, for free

This article will walk you through a quick-start of getting dbt-Core installed and run on your laptop within 5 minutes.

Pre-Requisites

Install pip in your machine.If you already have pip then check current pip version and update using :

pip --version  # Check Pip version
pip install --upgrade pip  #Upgrade to new version on pip

Install dbt-core on your local machine

Step #1: Create and Navigate to the directory where you want to setup dbt.

mkdir dbt

Step #2: To install dbt use this command.

pip install dbt-core

To install a specific adapter plugin use

pip install dbt-<adapter> 
# Example -> pip install dbt-postgres (This will install postgres adapter) 

Configure your first project

Step1: Initialize your project

Run this command to initialise the dbt project.

dbt init

This will:

• Create a new folder with your project name and sample files, enough to get you started with dbt

Create a connection profile on your local machine. The default location is ~/.dbt/profiles.yml.

After running the dbt init command, you'll have a new directory with a basic dbt project.

One of the files created is dbt_project.yml, which is where you can configure your project.

Open the dbt_project.yml file in a text editor and modify the name of your project (optional).

 name:'my_new_dbt_project'
 version:'1.0.0'

Step 2: Set up the database connection:

Next, you'll need to configure your database connection in the profiles.yml file. This file is in your user folder under ~/.dbt/.

Here's a simple example of what a Postgres profile might look like:

my_new_dbt_project:
  target: dev
  outputs:
    dev:
      type: postgres
      host: localhost
      user: my_user
      pass: my_password
      port: 5432
      dbname: my_database
      schema: my_schema

Replace my_user, my_password, my_database, and my_schema with your actual database credentials.

Step 3: Test Your Setup

Now that everything is set up, you can test your dbt project. Navigate to your project directory and run:

dbt debug

This command will test your connection to the database and verify that everything is working correctly. If there are any issues, dbt will provide you with information about what's wrong.

Step 4: Run your first dbt commands

Once everything is set up and tested, you can start using dbt!To compile your project, navigate to your dbt project directory and run:

dbt compile

To run your project, use the command:

dbt run 

And that's it! You've now set up dbt Core on your local machine. The process should have taken around 5 minutes, and you're now ready to start developing your data transformation workflows with dbt. Happy data modeling!