[spark][sparksql][odbc][jdbc] JDBC and ODBC drivers and configuration parameters

JDBC and ODBC drivers and configuration parameters

April 08, 2021

You can connect business intelligence (BI) tools to Databricks Workspace clusters and SQL Analytics SQL endpoints to query data in tables. This article describes how to get JDBC and ODBC drivers and configuration parameters to connect to Databricks Workspace clusters and SQL Analytics SQL endpoints. For tool-specific connection instructions, see Business intelligence tools.

Permission requirements

The permissions required to access compute resources using JDBC or ODBC depend on whether you are connecting to a Databricks Workspace cluster or SQL Analytics SQL endpoint.

Workspace cluster requirements

To access a cluster, you must have Can Attach To permission.

If you connect to a terminated cluster and have Can Restart permission, the cluster is started.

SQL Analytics SQL endpoint requirements

To access a SQL endpoint, you must have Can Use permission.

If you connect to a stopped endpoint and have Can Use permission, the SQL endpoint is started.

Prepare to connect BI tools

This section describes the steps you typically follow to prepare to connect to BI tools:

Step 1: Download and install a JDBC or ODBC driver

For some BI tools you use a JDBC or ODBC driver to make a connection to Databricks compute resources.

  1. Go to the Databricks JDBC or ODBC driver download page and download it. For ODBC, pick the right driver for your operating system.
  2. Install the driver. For JDBC, a JAR is provided which does not require installation. For ODBC, an installation package is provided for your chosen platform.

Step 2: Collect JDBC or ODBC connection information

To configure a JDBC or ODBC driver, you must collect connection information from Databricks. Here are some of the parameters a JDBC or ODBC driver might require:

Parameter Value
Authentication See Username and password authentication.
Host, port, HTTP path, JDBC URL See Get server hostname, port, HTTP path, and JDBC URL.

The following are usually specified in the httpPath for JDBC and the DSN conf for ODBC:

Parameter Value
Spark server type Spark Thrift Server
Schema/Database default
Authentication mechanism AuthMech See Username and password authentication.
Thrift transport http
SSL 1

Username and password authentication

This section describes how to collect the credentials for authenticating BI tools to Databricks compute resources.

When you configure authentication from your BI tool to Databricks resources the fields you set are labeled Username and Password. If SSO is enabled, set Username to the string token and Password to a personal access token. If SSO is disabled, you can either set Username to token and Password to a personal access token or set Username to your Databricks username and Password to your password.

Get server hostname, port, HTTP path, and JDBC URL

The procedure for retrieving JDBC and ODBC parameters depends on whether you are using Databricks Workspace clusters and SQL Analytics SQL endpoints.

Workspace cluster

  1. Click the Clusters Icon icon in the sidebar.

  2. Click a cluster.

  3. Click the Advanced Options toggle.

  4. Click the JDBC/ODBC tab.

    JDBC-ODBC tab
  5. Copy the parameters required by your BI tool.

SQL Analytics SQL endpoint

  1. Click the Endpoints Icon icon in the sidebar.

  2. Click an endpoint.

    경축! 아무것도 안하여 에스천사게임즈가 새로운 모습으로 재오픈 하였습니다.
    어린이용이며, 설치가 필요없는 브라우저 게임입니다.
    https://s1004games.com

  3. Click the Connection Details tab.

    Connection details
  4. Copy the parameters required by your BI tool.

Configure JDBC URL

The steps for configuring the JDBC URL depend on whether you are using a Databricks Workspace cluster or a SQL Analytics SQL endpoint.

Workspace cluster

In the following URL, replace <personal-access-token> with the token you created in Username and password authentication. For example:

 
jdbc:spark://<server-hostname>:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/0/xxxx-xxxxxx-xxxxxxxx;AuthMech=3;UID=token;PWD=<personal-access-token>

SQL Analytics SQL endpoint

Replace <personal-access-token> with the token you created in Username and password authentication. For example:

 
jdbc:spark://<server-hostname>:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/0/xxxx-xxxxxx-xxxxxxxx;AuthMech=3;UID=token;PWD=<personal-access-token>

Configure connection for native query syntax

JDBC and ODBC drivers accept SQL queries in ANSI SQL-92 dialect and translate the queries to Spark SQL. If your application generates Spark SQL directly or your application uses any non-ANSI SQL-92 standard SQL syntax specific to Databricks, Databricks recommends that you add ;UseNativeQuery=1 to the connection configuration. With that setting, drivers pass the SQL queries verbatim to Databricks.

Configure ODBC Data Source Name for the Simba ODBC driver

The Data Source Name (DSN) configuration contains the parameters for communicating with a specific database. BI tools like Tableau usually provide a user interface for entering these parameters. If you have to install and manage the Simba ODBC driver yourself, you might need to create the configuration files and also allow your Driver Manager (ODBC Data Source Administrator on Windows and unixODBC/iODBC on Unix) to access them. Create two files: /etc/odbc.ini and /etc/odbcinst.ini.

/etc/odbc.ini

  1. Set the content of /etc/odbc.ini to:

    ini
    [Databricks-Spark]
    Driver=Simba
    Server=<server-hostname>
    HOST=<server-hostname>
    PORT=<port>
    SparkServerType=3
    Schema=default
    ThriftTransport=2
    SSL=1
    AuthMech=3
    UID=token
    PWD=<personal-access-token>
    HTTPPath=<http-path>
    
  2. Set <personal-access-token> to the token you retrieved in Username and password authentication.

  3. Set the server, port, and HTTP parameters to the ones you retrieved in Get server hostname, port, HTTP path, and JDBC URL.

/etc/odbcinst.ini

Set the content of /etc/odbcinst.ini to:

ini
[ODBC Drivers]
Simba = Installed
[Simba Spark ODBC Driver 64-bit]
Driver = <driver-path>

Set <driver-path> according to the operating system you chose when you downloaded the driver in Step 1:

  • MacOs /Library/simba/spark/lib/libsparkodbc_sbu.dylib
  • Linux (64-bit) /opt/simba/spark/lib/64/libsparkodbc_sb64.so
  • Linux (32-bit) /opt/simba/spark/lib/32/libsparkodbc_sb32.so

Configure paths of ODBC configuration files

Specify the paths of the two files in environment variables so that they can be used by the Driver Manager:

ini
export ODBCINI=/etc/odbc.ini
export ODBCSYSINI=/etc/odbcinst.ini
export SIMBASPARKINI=<simba-ini-path>/simba.sparkodbc.ini # (Contains the configuration for debugging the Simba driver)

where <simba-ini-path> is

  • MacOS /Library/simba/spark/lib
  • Linux (64-bit) /opt/simba/sparkodbc/lib/64
  • Linux (32-bit) /opt/simba/sparkodbc/lib/32

 

[출처] https://docs.databricks.com/integrations/bi/jdbc-odbc-bi.html

 

 

 

본 웹사이트는 광고를 포함하고 있습니다.
광고 클릭에서 발생하는 수익금은 모두 웹사이트 서버의 유지 및 관리, 그리고 기술 콘텐츠 향상을 위해 쓰여집니다.
번호 제목 글쓴이 날짜 조회 수
공지 오라클 기본 샘플 데이터베이스 졸리운_곰 2014.01.02 86136
공지 [SQL컨셉] 서적 "SQL컨셉"의 샘플 데이타 베이스 SAMPLE DATABASE of ORACLE 가을의 곰을... 2013.02.10 78637
공지 [G_SQL] Sample Database 가을의 곰을... 2012.05.20 95363
944 [java dbms][database] [컴] Apache Derby 사용하기 - 1 - Derby 설치 file 졸리운_곰 2021.04.15 1153
» [spark][sparksql][odbc][jdbc] JDBC and ODBC drivers and configuration parameters file 졸리운_곰 2021.04.14 4127
942 [spark][pyspark][php] Natively Connect to Spark Data in PHP 졸리운_곰 2021.04.14 1702
941 [데이터분석][python] Dash를 사용하는 초보자 및 기타 모든 사용자를위한 Python의 대시 보드 file 졸리운_곰 2021.04.14 1762
940 [데이터분석][python] Dash를 사용하는 초보자 및 기타 모든 사용자를위한 Python의 대시 보드 file 졸리운_곰 2021.04.14 1602
939 [sqlite] SQlite source code analysis-architecture file 졸리운_곰 2021.04.12 1807
938 [SQLite] SQLite 사용자 함수 추가 졸리운_곰 2021.04.12 1663
937 {SQLite] SQLite 페이지 핸들링(3) - 레코드 포맷 졸리운_곰 2021.04.12 1667
936 [SQLite] SQLite 페이지 핸들링(2) - SQLite의 페이지 포맷 file 졸리운_곰 2021.04.12 1496
935 [SQLite] SQLite 페이지 핸들링(1) - SQLite의 구조 file 졸리운_곰 2021.04.12 1486
934 [C/C++ 자료구조] SQLite 의 모든 것 (4부) - Java 에서 사용하기 Database/SQLite file 졸리운_곰 2021.04.12 1662
933 [C/C++] SQLite 의 모든 것 (3부) - C++ 에서 사용하기 Database/SQLite 졸리운_곰 2021.04.12 1719
932 [C/C++ 자료구조] SQLite 의 모든 것 (2부) - Download & Build Database/SQLite file 졸리운_곰 2021.04.12 1665
931 [C/C++ 자료구조] SQLite 의 모든 것 (1부) - 소개 및 FAQ Database/SQLite file 졸리운_곰 2021.04.12 1332
930 [NoSQL] [Redis] Redis Persistence(영속성) 졸리운_곰 2021.04.11 1644
929 [NoSQL] [Cloud] Redis 설치, 사용 방법, 데이터 백업을 위한 RDB & AOF 개념 및 간단한 Redis 사용 사례 연구 file 졸리운_곰 2021.04.11 1716
928 [SPARK][Python][pySpark][아콘 소프트][나무기술] How to Run a Spark Standalone Job 졸리운_곰 2021.04.05 982
927 [SPARK][Python][pySpark][아콘 소프트][나무기술] Real-world Python workloads on Spark: Standalone clusters : 스파크 예제 논란, driver-host 불필요 file 졸리운_곰 2021.04.03 1691
926 [데이터분석][데이터 사이언스][python][Dash] Python, Dash 및 Plotly를 사용하여 COVID-19 사례 데이터 시각화 file 졸리운_곰 2021.03.28 1501
925 [데이터분석][머신러닝] When not to use machine learning or AI Adventures in wishful thinking, nonstationarity, and pattern-finding / 기계 학습 또는 AI를 사용하지 않아야하는 경우 희망찬 사고, 비정상 성, 패턴 찾기의 모험 file 졸리운_곰 2021.03.28 21621
대표 김성준 주소 : 경기 용인 분당수지 U타워 등록번호 : 142-07-27414
통신판매업 신고 : 제2012-용인수지-0185호 출판업 신고 : 수지구청 제 123호 개인정보보호최고책임자 : 김성준 sjkim70@stechstar.com
대표전화 : 010-4589-2193 [fax] 02-6280-1294 COPYRIGHT(C) stechstar.com ALL RIGHTS RESERVED