Databricks Connect on Windows

This post takes you through setting up the Databricks Connect client on windows and is a requirement for subsequently working with a Visual Studio environment.

The case of this example, the cluster is running Databricks 6.5 so all commands will reflect this. I have assumed that you are able to create a Workspace, this is a prerequisite.

Create and Start a Databricks Cluster

For the sake of this example I have create the simplest, single node cluster possible, without auto scaling

Sample 6.5 Cluster

Once it is created and running, select the cluster and copy the URL:

Sample Cluster URL

The key information to note is highlighted above and translates as follows:

Organisation Id1234567891011121314
Cluster Id0621-093110-abcd987
Key URL Values

Obtain a Token

Now that we have a URL, Organisation Id and Cluster Id we finally need a Token before we can install Databricks Connect and then configure it. This is achieved by navigating to your account in Databricks Workspace

Navigating to create a Token

Select the option to create a New Token, make sure this is noted as you will need it later

Creating a Token

You will be prompted to record the token for future use. The token will be similar to

89E91D0748A846A08853E961E2242832

Install Databricks Connect

We now have the basic details to install and configure Databricks Connect. From the command line run the following:

pip install -U databricks-connect==6.5.*

Configure Databricks Connect

Once Databricks Connect is installed, run the following from the command line

databricks-connect configure

You will now be prompted to enter the following details we have already captured:

Do you accept the above agreement? [y/N] y

Set new config values (leave input empty to accept default):

Databricks Host [no current value, must start with https://]: <databricks-url>

Databricks Token [no current value]: <databricks-token>

Cluster ID (e.g., 0921-001415-jelly628) [no current value]: <cluster-id>

Org ID (Azure-only, see ?o=orgId in URL) [0]: <org-id>

Port [15001]: <port>

The following are the sample values previously collected

URLhttps://adb-1234567891011121314.12.azuredatabricks.net/?o=1234567891011121314#/setting/clusters/0621-093110-abcd987
Token89E91D0748A846A08853E961E2242832
Organisation Id1234567891011121314
Cluster Id0621-093110-abcd987
PortSimply us the default value of 15001
Sample Values for configuring Databricks Connect

Test Databricks Connect

We have now configured Databricks Connect. From the command line run the following to test:

databricks-connect test

If all proceeds as expected, you will ultimately be presented with an output similar to the following:

All tests passed.

Ok we are now good to go!!

Additional Reading

https://docs.databricks.com/dev-tools/databricks-connect.html

Leave a Reply

Your email address will not be published. Required fields are marked *

RSS
LinkedIn
Close Bitnami banner
Bitnami