Combining node.js and Python


Question

Node.js is a perfect match for our web project, but there are few computational tasks for which we would prefer Python. We also already have a Python code for them. We are highly concerned about speed, what is the most elegant way how to call a Python "worker" from node.js in an asynchronous non-blocking way?

1
123
5/27/2012 4:03:52 PM

Accepted Answer

For communication between node.js and Python server, I would use Unix sockets if both processes run on the same server and TCP/IP sockets otherwise. For marshaling protocol I would take JSON or protocol buffer. If threaded Python shows up to be a bottleneck, consider using Twisted Python, which provides the same event driven concurrency as do node.js.

If you feel adventurous, learn clojure (clojurescript, clojure-py) and you'll get the same language that runs and interoperates with existing code on Java, JavaScript (node.js included), CLR and Python. And you get superb marshalling protocol by simply using clojure data structures.

84
6/29/2014 9:16:20 AM

This sounds like a scenario where zeroMQ would be a good fit. It's a messaging framework that's similar to using TCP or Unix sockets, but it's much more robust (http://zguide.zeromq.org/py:all)

There's a library that uses zeroMQ to provide a RPC framework that works pretty well. It's called zeroRPC (http://www.zerorpc.io/). Here's the hello world.

Python "Hello x" server:

import zerorpc

class HelloRPC(object):
    '''pass the method a name, it replies "Hello name!"'''
    def hello(self, name):
        return "Hello, {0}!".format(name)

def main():
    s = zerorpc.Server(HelloRPC())
    s.bind("tcp://*:4242")
    s.run()

if __name__ == "__main__" : main()

And the node.js client:

var zerorpc = require("zerorpc");

var client = new zerorpc.Client();
client.connect("tcp://127.0.0.1:4242");
//calls the method on the python object
client.invoke("hello", "World", function(error, reply, streaming) {
    if(error){
        console.log("ERROR: ", error);
    }
    console.log(reply);
});

Or vice-versa, node.js server:

var zerorpc = require("zerorpc");

var server = new zerorpc.Server({
    hello: function(name, reply) {
        reply(null, "Hello, " + name, false);
    }
});

server.bind("tcp://0.0.0.0:4242");

And the python client

import zerorpc, sys

c = zerorpc.Client()
c.connect("tcp://127.0.0.1:4242")
name = sys.argv[1] if len(sys.argv) > 1 else "dude"
print c.hello(name)

Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Icon