Our Training Programs

All contents are copyright of their authors.

How to read Azure Databricks output using API or class library ?

How to read Azure Databricks output using API or class library ?

Prashant Rewatkar

5yAsked Sep 23, 2021, 4:38 AM1k.NET

I have Azure Databrick notebook which contain SQL command. I need to capture output of SQL command and use in Dot Net core. Need help.

Know the answer? Post it — somebody with the same question will find it here.

It is the same account you read, post and publish with — and you will come straight back to this page.

Srinivasan Ramamoorthi5y ago
Accepted answer
In that case, I would recommend to use Azure Data Factory to invoke the notebook and extract the output to SQL tables. You can then read the data from SQL tables.
Prashant Rewatkar5y ago
Fine with ADF. but i am looking for direct API call/execute using C# where we will pass sql command and other command as well which we use to call generally using ADB. I dont want to go on Azure portal/ADF/ Any other platform to execute code. All action will get perform in Dot net core envierment.
Prashant Rewatkar5y ago
Thanks for replay, but unfortunetly there is no API to get output of Notbook. You could get exported HTML but not exaxt output like table. for example you are execute "SELECT * FROM Table" and output is table. This table i need. I checked those API earler but no success.
Srinivasan Ramamoorthi5y ago
You can refer this article to invoke Databricks REST API from Python and Powershell. Similarly .NET core should be also supporting the REST APIs

https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/