Oracle introduces GraphPipe for Machine Learning Deployment

Recently, Oracle has introduced GraphPipe, an open source framework, to simplify and standardize machine learning model deployment.

Recently, Oracle has introduced GraphPipe, an open source framework, to simplify and standardize machine learning model deployment.
 
In the absence of a standard protocol for how tensor-like data should be transmitted over a network. The organizations have been facing following three challenges while deploying machine learning model into a project.
  1. No standard exists for model serving APIs, general application are using a bespoke approach to access the deployed model.
  2. To develop a model server for your organization is incredibly complicated as there are not enough easy to use tools.
  3. Commonly utilized approaches do not focus on performance. 
"We created GraphPipe to solve these three challenges. It provides a standard, high-performance protocol for transmitting tensor data over the network, along with simple implementations of clients and servers that make deploying and querying machine learning models from any framework a breeze. GraphPipe's efficient servers can serve models built in TensorFlow, PyTorch, mxnet, CNTK, or caffe2." posted Vish Abrams, architect for cloud development at Oracle.
 
Presently industry train machine learning model individually to different sections while deployment and access to network among different servers is done using bespoke methods. This impacts the organization's ability to derive value from its investment. The custom deployment is often at risk of getting cracked under the load. GraphPipe contains the best possible tools that will enable the enterprise to derive value from its deployment investment.
 
The GraphPipe open source package includes a set of FlatBuffer definitions, Guidelines for serving models based on FlatBuffer definitions, Examples for serving models from various machine learning frameworks, client libraries for querying models.
 
According to Oracle, GraphPipe is based on flatbuffers, using flatbuffer as the message format helps avoid a memory copy during deserialization phase.
 
 
 Source: Oracle
 
 
 
 Source: Oracle
 
 
To read full documentation and examples on GraphPipe you can visit Oracle. To find GraphPipe flatbuffer spec and servers for Python and Go you can visit Oracle's GitHub.