javascript - Apache Spark and node.js - Stack Overflow

admin管理员组

文章数量:1334194

I would like to call node.js program from spark-submit, using Spark API within JavaScript. Is there any implementation of this API in JavaScript?

I know one can call external mapper and reducer (written in JavaScript) from Hadoop, but I want to be able to run the plete program as one job from Spark.

I would like to call node.js program from spark-submit, using Spark API within JavaScript. Is there any implementation of this API in JavaScript?

I know one can call external mapper and reducer (written in JavaScript) from Hadoop, but I want to be able to run the plete program as one job from Spark.

• javascript
• node.js
• apache-spark

Share Improve this question Follow asked Jun 30, 2015 at 17:18 Diamond DutchDiamond Dutch 5111 silver badge55 bronze badges 1

• There's github./syzer/JS-Spark – Andrew Commented Jan 22, 2016 at 1:34

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 4

There is a new project called EclairJS that is providing a Spark API from Node and JavaScript, see https://github./EclairJS/eclairjs-node.

I don't believe there is a current JavaScript spark wrapper, however I'd check out the information on the Spark pipe interface, this will allow you to use your JavaScript code from within Spark. Best of luck with your Spark & Javascript adventures (sadly there was an april fools joke about Spark & ScalaJS which makes searching for information on Spark + Javascript a little bit tricky).

本文标签： javascriptApache Spark and nodejsStack Overflow