.NET For Apache Spark Version 1.0 Released

Version 1.0 of .NET for Apache Spark brings support for .NET applications targeting .NET Standard 2.0 or later, and access to the Apache Spark DataFrame APIs.

Recently, Microsoft announced the release of version 1.0 of .NET for Apache Spark, which is an open-source package that brings .NET development to the Apache Spark platform.

The release includes support for .NET applications targeting .NET Standard 2.0 or later, and access to the Apache Spark DataFrame APIs. The release also brings the ability to write Spark SQL and create user-defined functions (UDFs).

Source: Microsoft

Well, .NET for Apache Spark was introduced two years ago to address the increasing demand from the .NET community for an easier way to build big data applications. Microsoft says that the biggest motivation to use the package is to take advantage of existing .NET development skills and resources, including the enormous .NET ecosystem of existing libraries and frameworks. 

The latest version 1.0 also gives users an API extension framework to add support for additional Spark libraries. Currently, this includes Linux foundation Delta Lake, Microsoft OSS Hyperspace, ML.NET, and support for Apache Spark’s MLLib functionality

Microsoft said that it is committed to the continuous evolution of the product to integrate the latest features and keep the API current with the latest Spark versions. 

To get started you can visit the tutorial Get started with .NET for Apache Spark.