Multi-language Flowex components

In Flow-Based Programming (FBP) paradigm each component is an independent process — “black box”. Component only transforms input to output, and this transformation is independent of internal implementation. So in general components can be written in different languages and then linked together in a single network.

Ability to run alias code is very important for such new languages as Elixir. One can reuse tonnes of libraries from more mature languages and choose the specific language for the specific problem.

In this article, I’ll show examples of multi-language Flowex components. I will use Export library to run Ruby and Python and Porcelain library to run command-line applications. There is the multi_flowex project on GitHub, for whom interesting in details.

Erlang ports

Erlang port is an abstraction that provides a byte-oriented interface to an external program over STDIN and STDOUT by sending and receiving lists of bytes, including binaries. Ports are process specific, so only process created a port can communicate with the port and if this process terminates, so does the port. The external program resides in another OS process. By default, it reads from standard input (file descriptor 0) and writes to standard output (file descriptor 1). The external program is to terminate when the port is closed.

Erlport project is not in active development but demonstrates how one can connect Erlang to other programming language. Currently, the project supports only Ruby and Python. There is a wrapper for Elixir — Export.

Porcelain allows launching and communicating with external programs.

multi_flowex

multi_flowex is a simple Flowex project with Ruby, Python and shell “pipes”. Each component says “Hello”. So passing an information packet (IP) with empty input return IP with the text: “Hello from Ruby, Hello from Python, Hello from shell”.

Let’s consider each component in details. (Read more about Flowex module pipes here)

Ruby pipe

The init function starts Ruby process and adds its pid to options. @ruby_dir specifies the path to ruby code and @main_file the file with simple Ruby code:

It just pushes element to array.

The code in the call function of the component calls the push method with initial data and “Hello from Ruby”.

Take a look at the test of the module to get the idea what each function returns.

Python pipe

The pipe with Python code looks similar:

And the Python code is:

Shell pipe

A component which uses Porcelain just runs echo program and gets its output:

The pipeline

Let’s look at the pipeline itself.

The pipeline starts 3 Ruby pipes, 4 Python and 2 Shell pipes. It’s just for demonstration purposes, we definitely don’t need such parallelism is not necessary for this “Hello, world!” example. One can check that after starting the pipeline 3 Ruby and 4 Python processes appears in the system. Porcelain starts `echo` process only on demand, when IP enters the component.

The last pipe is plain Elixir pipe which just joins a list into final result string:

The spec below shows the pipeline in action:

So one can see greetings from 3 different programs!

Conclusion

Elixir/Erlang ecosystem has a flexible and safe way of running external programs thus allowing to leverage the advantages of other programming languages. At the same time, Flowex makes it easy to encapsulate alias programs inside independent processes and organize communication between them.

Software engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store