I've just started reading Joe Armstrongs book on Erlang and listened to his excellent talk on Software Engineering Radio.
Its an interesting language/system and one whose time seems to have come around with the advent of multi-core machines.
My question is: what is there to stop it being ported to the JVM or CLR? I realise that both virtual machines aren't setup to run the lightweight processes that Erlang calls for - but couldn't these be simulated by threads? Could we see a lightweight or cutdown version of Erlang on a non Erlang VM?
You could not use JVM/CLR libraries, given their reliance on mutable objects.
Erlang exception handling is quite different from JVM and CLR exceptions, you would need to handle this somehow.
Implementing processes as threads would mean that any sizable Erlang system runs out of memory pretty fast (process size on my machine on creation: 1268 bytes, thread stack size in CLR: 1 MB) and communication between processes is much slower than in Erlang.
What you probably want is an Actor Model implementation on JVM or CLR.
Scala and Clojure have already been mentioned. In addition, there are many Actor implementations for JVM:
Kilim, Functional Java, Jetlang, Actors Guild, ActorFoundry, and at least one for CLR: Retlang, which can be used from any JVM/CLR language.
For educational reasons, we are implementing a subset of ErlangVM for CLR. We were highly inspired by Kresten Krab Thorup and his project Erjang, a JVM based Erlang VM. Erjang uses kilim framework for representing lightweight processes, and it starts to attract attention.
Javalimit - Erjang's author blog.
Erjang repository
This is a well trod-discussion. Some context might be useful.
From the Erlang mailing list last November:
The start of a long discussion thread
continuing here
and going a bit mental here
and ending on Joe's contribution.
My contribution to the debate about Erlang on the JVM? No, not a good idea :(
Nothing at all, actually. You might have a look at Clojure, which is an interesting functional language built on the JVM.
Axum -- an incubation project on the CLR -- was clearly inspired by Erlang.
Erjang is a virtual machine for Erlang, which runs on Java™.
I don't know of any technical problem inhiting this.
Actually Scala (a JVM functional language) uses what is called an Actor Model that is very similar to, and as I understand it borrows heavily from, the Erlang model of shared-nothing concurrency.
Threads could not simulate Erlang processes. They're much too heavy-weight.
Just for completeness additional source about topic.
Possible? Yes. Practical? Well, probably not; they solve different problems in very different ways, and thus have lots of major differences in the way they do things. This would make porting hard, and performance would likely suffer severely. That doesn't mean it can't be done, just that there are better ways to accomplish what such a port would bring to the table.
Related
I was creating distributed systems in OOP languages using message passing libraries like MPI, ZepoMQ, RabbitMQ and so on. Now I found myself watching some erlang promotional material and understood that lots of things we emulate in OOP languages like C++ and C# using libraries (1 000 000 socket connections per process, distributed messaging and distributed process monitoring visualization) was there in Erlang for many years now. And it seemed reasonable to get to know the language better. I found myself asking one last question: are there any implementations\prototypes of Erlang alike VM that could run/spawn some processes not only on CPU but also on GPU?
Because that would definitely make Erlang (and its more readable for my OOP background dialects like Elixir) language of choice for most future projects.
GPU is fast only with sequential memory access. I hardly imagine garbage collection on GPU RAM. GPU is NOT a cool and parallel CPU. It requires more effort to write to. So most probably there is no Erlang compiler for GPU.
I doubt there's any implementation that can run Erlang processes on GPU but you can use two techniques to run computations on GPU under Erlang:
use C library through NIFs (native implemented functions) - see http://www.erlang.org/doc/man/erl_nif.html and an example of such an implementation: msantos/procket on Github (I'm sorry, I can't post the link due to low reputation :)
use native OS process and communicate with it through erlang "port" - see http://www.erlang.org/doc/reference_manual/ports.html
The first one is faster and the later is safer (NIFs can crash the whole VM).
This is not specific to GPU coputations. Erlang is not well suited for high performance number crunching - it's better to do it in C and manipulate the results in Erlang anyway. The communication between the C and Erlang should be implemented in the one of the two described manners.
I'm starting to dive into Erlang for the first time, and OTP is held aloft by lovers and critics alike as being the gold standard for highly available, distributed processing.
Given that OTP has been around for decades and is openly documented, why is it that other languages supporting lightweight threads/processes haven't adopted versions of their own? Are there technical/political challenges? Or does everyone just shrug and learn Erlang?
Thanks!
The largest issue is that most language runtimes don't have built-in lightweight concurrency and error isolation with exit signal propagation. Without those things you would have a really hard time properly porting OTP.
For the languages that do have the right kind of runtime, I am seeing some effort or at least plans to build OTP inspired frameworks. Cloud Haskell is the first that comes to mind. I also expect that Go and Rust will eventually have something like OTP if they don't already.
There are technical challenges, as Erlang itself is designed for the same features OTP is known for. Case in point, Basho Riak is a distributed fault-tolerant key/value store written in Erlang. One might be able to port it to Haskell or some similar functional language, but it would probably be a lot of work. Just for fun, you might look into OTP stuff written in the Elixir language.
Actually, it has been (tried).
Akka is the library which takes some OTP features and implements them in Scala for JVM.
Given the principles underlying JVM and BEAM (the Erlang VM) are very different (mainly GC, scheduling and message passing are radically different), I can't say how successful that implementation is and how many benefits of the original OTP it preserves. There's a lot of (heated) debate on that in the internets.
I want to write actor style code for embedded processors and I am trying to decide between writing everything in Erlang vs writing everything in zeromq+any language. Using zeromq looks to be very powerful in the sense that I can use any programming language and make my development a lot easier(many available libs) but then I am not sure if there is any gotcha in this power? I understand that Erlang represent actor model much better especially with OTP concepts but then it seems easy to represent similar actor model with zeromq? Am I looking at this correctly?
1.What do I really lose not using Erlang for embedded applications (where distributed processing, a power point of Erlang, is NOT required) and just build things on top a generic messaging framework like zeromq?
2.Is Erlang offering more than a coordinated messaging framework for a non-distributed embedded application?
3.What specific capabilities of Erlang could took too long to implement with zeromq?
You're comparing apples and oranges. Part of the advantage of using Erlang is the language; if you're going to put it up against zmq + some other language, the other language in that comparison really matters. zmq + ARM assembly? Erlang brings all the wonderful advantages of not hand-coding ASM.
As for what else Erlang brings to the table, Embedded Erlang? Absolutely argues that Erlang has advantages in fault tolerance, hot code loading, rapid development by leveraging Erlang and OTP, easy interaction with C libraries, and simple debugging by live REPL and copy-paste of terms.
Some of those things, such as hot reload, on-device REPL, and established libraries, will definitely take some real hacking to reproduce from the ground up.
My point would be that you will have to work very hard to get the same kind of error handling in Zmq. Erlang has some really nice built-in error handling when things begins to go bad. There has been considerable time spent in Erlang optimizing that part and making it robust.
Zmq on the other hand, is probably faster in some combinations with some languages when you make simple benchmarks. There is less overhead, so it may process messages faster than what Erlang can provide.
But chances are that you will end up re-implementing large parts of Erlang in the language of your choice. And you will probably not do a job as good as 6-10 developers working on Erlang/OTP for 15 years.
On the other hand, Erlang is not a simple language to learn. There is way more to it than just learning how to program in a functional style. Especially the concurrency patterns and failure handling can take some time getting used to.
ZeroMQ =/= Erlang covers many differences. The claim there is that ZeroMQ only provides the messaging aspect, not the light-weight processes, process monitoring and other aspects.
I think Erlang is very well suited for server systems developed in my workplace (currently developed in Java). I am a bit skeptical how this would be accepted both by developers (who have no idea about functional or Erlang) and by managers.
Any ideas on how to approach the issue? I am thinking about some hybrid system, where the hardcore highly reliable infra uses Elrang, and app specific stuff developed in Java (as nodes?)
There are a few approaches, and neither have any guarantees to actually work
Implement something substantial in a short time frame, perhaps using your own time. Don't tell anyone until you have something to display that works. Unless you have a colleague in on it.
Pull up lots of Erlang projects that are good demonstrations of the features you want. Present it to your managers and try to frame them about the risk in keeping using Java with this kind of technology available.
If the company you work for actually have a working code base in Java already, they're not likely to take you seriously when you suggest to rewrite it in another language.
The true test that you believe in Erlang being a much better choice: Quit and start up a competing company and bring the technology insight you have in your current industry. Your managers are really comparing a similar risk-scenario as you would do if you were to quit your job, and they are looking for the same assuring facts for success as you would do, to consider leaving a "safe" paycheck.
As for how to integrate, check out the jinterface application in Erlang. It allows Java code to send messages to Erlang nodes, and it allows Java to expose mailboxes to the Erlang nodes as if there were Erlang processes.
It's all about ROI (Return On Investment) to a manager: a manager will be concerned about performance (of the company). In order to appeal to his business nature, you'll have to make a case for it using dollar$ (or whatever appropriate currency).
Beware that undertaking a "skunkwork" project on the side to "prove" your solution based on Erlang might backfire: "so you had time to play with Erlang, why didn't you spend the time on the project then?" (Of course, not all managers/companies would think this way).
You have to take into account the whole proposal e.g. impact on the team, skills to be developed etc. It's all about money.
If I have an advice for you: start small, plant a seed, nurture it and watch it grow.
A wise man once said to me:
"It's not about technology, it's about
the product & market".
Start by not targetting a rewrite but using erlang for a new feature/project. Rewrites can be expensive and taking a chance on erlang for something that is already a time consuming and costly undertaking is a hard sell. But if there is a new piece that could be done in erlang and java, you stand a better chance. The project will be small enough hopefully that you can discover early if erlang is a good fit and adapt accordingly. And when erlang proves itself in that project you will have better data to make your case with.
We're introducing RabbitMQ into our infrastructure, which currently runs a combination of C++, Java and Python applications. I'm not specifically intending to move the team towards Erlang, but if I were, introducing a well-written third-party tool that just happens to use Erlang is a very good way to get the foot in the door.
One major caveat is that while Erlang is a wonderful language to learn, the surrounding technology (OTP in particular) has a huge learning curve and is extremely primitive in many ways (debugging, IDE's, etc.). It is getting better all the time, but reluctant converts will crucify you if you don't warn them about the pain of learning to program in a radically different environment. Even simple things like the lack of code-sense technology (E.g., type 'foo.' and the IDE tells you what methods you can call on foo) can leave a really bad taste in the mouth.
Erlang is getting a reputation for being untouchable at handling a large volume of messages and requests. I haven't had time to download and try to get inside Mr. Erlang's understanding of switching theory... so I'm wondering if someone can teach me (or point to a good instructional site.)
Say as a thought-experiment I wanted to port the Erlang ejabberd to a combination of Python and C, in a way that gave me the same speed and scalability. What structures or patterns would I have to understand and implement? (Does Python's Twisted already do this?)
How/why do functional languages (specifically Erlang) scale well? (for discussion of why)
http://erlang.org/course/course.html (for a tutorial chain)
As far as porting to other languages, a message passing system would be easy to do in most modern languages. Getting the functional style can be done in Python easily enough, although you wouldn't get the internal dispatching features of Erlang "for free". Stackless Python can replicate much of Erlang's concurrency features, although I can't speak to details as I haven't used it much. If does appear to be much more "explicit" (in that it requires you to define the concurrency in code in places that Erlang's design will allow concurrency to happen internally).
Erlang is not only about scalability but mostly about
reliability
soft real-time characteristics (enabled by soft real-time GC which is possible because immutability [no cycles] and share nothing and so)
performance in concurrent tasks (cheap task switch, cheap process spawn, actors model, ...)
scalability - debatable in current state , but rapidly evolving (about 32 cores well, it is better than most competitors but should be better in near future).
Another of the features of erlang that have an impact on scalability is the the lightweight cheap processes. Since processes have so little overhead erlang can spawn far more of them than most other languages. You get more bang for your buck with erlang processes than many other languages give you.
I think the best choice for Erlang is Network bound applications - makes communication much simpler between nodes and things like heartbeat monitoring, auto restart using supervisor are built into OTP.