I am currently learning Erlang
Can SO users give interesting examples of any of their Erlang application deployments?
I want to gain some insight into common Erlang uses past telecomms, and any problems or unexpected benefits Erlang brought during development/deployment.
I hope this will give some broader context and whet the whistle for myself and anyone else jumping into Erlang!
Thanks in advance!
Who uses Erlang for product development:
Bluetail/Alteon/Nortel (distributed,
fault tolerant email system, SSL
accelerator)
Cellpoint (Location-based Mobile
Services)
Corelatus (SS7 monitoring).
dqdp.net (in Latvian) (Web Services).
Facebook (Facebook chat backend)
Finnish Meteorological Institute
(Data acquisition and real-time
monitoring)
IDT corp. (Real-time least-cost
routing expert systems)
Klarna (Electronic payment systems)
Mobilearts (GSM and UMTS services)
Netkit Solutions (Network Equipment
Monitoring and Operations Support
Systems)
Process-one (Jabber Messaging)
Schlund + Partner (Messaging and
Interactive Voice Response services)
Quviq (Software Test Tool)
RabbitMQ (AMQP Enterprise Messaging)
T-Mobile (previously one2one)
(advanced call control services)
Telia (a telecomms operator)
Vail Systems (Interactive Voice
Response systems)
Wavenet (SS7 and IVR applications)
Our first application was a web/sms social network and I wrote a long paper on the subject which can be read here.
We've built a web app based on an Erlang backend.
Erlang is in charge of the business logic, the security and data store.
The browser communicates exclusively through JSON services with it and do the rendering.
It will be in beta soon, and to give you an idea of the app there is a video here
There are as well some resources here and here about what we learned along the way.
Get to know the the release tools erlang/OTP already provides.
Erlang bootscripts are wonderful for ensuring that all the running applications needed are present and of the correct version. Working within the OTP framework for releases will be much easier than trying to invent your own. Erlang has lots of tools for making sure deployments can be done both live and without breaking running services. The language and runtime are designed for this so they've done a lot of the heavy lifting for you. I've found the tools useful even for small "non-enterprise" apps and deployments.
Of course there's always applications like wings3D which is for 3D modelling. It's not exactly a "deployment", because these sorts of programs are used anywhere from individuals to teams of artists in their pipeline. There are other projects for things like simulation, but I'm not sure how many companies are publicly stating that they use Erlang. As for me, I'm planning to adopt it for my company for industrial automation.
Related
MEAN stack and IOT are the current trending hot topics. Can these two be used together? If yes then in what way?
How can these technologies be used together?
Sweta.
By saying MEAN.js you are including things that are not strictly in the IoT terrain. Angular for example has little to do with anything.
On the web front end you need to implement a javascript library like Paho.js that will use the MQTT protocol to connect to a broker and start aggregating messages from connected devices.
Express has little to do as well as you are not exposing a Restful interface but connecting low level through a broker. A good solution in Node.js is Mosca.
Mongo is good for dumping data from devices.
I have written a tutorial using Node.js and iOS so have a look and you might find it interesting.
Mean stack is the combination of the frontent web frameworks like angularjs,emberjs,knockoutjs,backbonejs , javascript's backend server called nodejs and using the mongodb at top. so using these frameworks and library will make a mean stack developer.
IoT pronounced as Internet of things. iot is recently used for connected electronics devices .basically it is a form of running your program inside the electronic chip and mostly trying to connect the devices.making control on devices using the programmed chip. there are separate IDE's are avaialble for developing and testing the programme on embeded chip.
you can use angularjs as a frontend(making your GUI) for your IoT'S application.
Yes, you can.
As a fact is has been done before. And in other frontend frameworks too. Here there is an example for home automation.
You can find even a yeoman generator for such projects here.
[Disclaimer: I work here] Netbeast started managing devices and creating a system of plugins on top
of a MEAN app and RESTful communications. (Now we use a MERN
stack, with react and MQTT over websockets to control networks and
update values in real time.)
To mention other places where you can find examples of current projects using MEAN to run IoT networks I encourage you to join angular, arduino and raspberry communities, as well as taking a tour over producthunt.com, hackster.io and other maker sites such as the previously mentioned Netbeast forum.
Yes you can make an IoT platform with the MEAN stack. Typically the sensors are low cost sensors and are constantly transmitting small amounts of data in MQTT or TCP protocols. With Node.js you can write, servers for such applications very easily.
Mongo is useful if you have unstructured data, which could happen if you work with multiple sensors. If you don't need unstructured data structures, SQL is sufficient.
All the data that you get from devices, finally needs to be consumed via applications. Express and Angular are great platforms to manage web applications.
You can read a little more about IoT platforms in MEAN at http://blog.yatis.io/scalable-iot-platform-mean-stack/
This is a somewhat general question but I'm hoping someone will have specific info or recommendations.
I have an application suite that includes a service application that acts as a communications interface and data historian for industrial pollution-control hardware. The service contains a singleton COM server to allow the rest of the suite to have access to the hardware and data via the service.
I've read the stuff about how SvCom is required to make COM servers work in Delphi service apps. I have and use SvCom - it does what it claims. But I'm not all that comfortable with it, the product and my coding styles and expectations don't match, and it makes debugging somewhat more of a headache.
But my real problem is with the idea that the lengths SvCom goes to to make a COM server work in a service app is absolutely required. Their documentation, and some of the stuff that comes up in searches on the subject, makes it sound like their toolbox is absolutely required for any COM-server-in-service scenario. But I have a couple of different 3rd-party libraries for implementing OPC servers, Prosys Sentrol and the older Production Robots library (if you're not familiar with OPC, it's a pretty-much ubiquitous data-interchange standard built on COM) and both support the OPC COM server being put in standard TService-based apps without special handling beyond doing in the AfterInstall and BeforeUninstall events what would normally be done in a stand-alone EXE when run with the /regserver or /unregserver command-line switches, and of course using DelayInitialize := True. So at least SOME COM servers can be done as typical TService-based apps without the extraneous steps SvCom goes through.
So my question is: Is the line between "what sorts of COM servers work in a TService-based app" and "what causes the need for the extra stuff SvCom does" clearly known? If so, what is it and/or where is it documented? If not, I'm kind of surprised - seems like implementing COM servers in service apps would be a fairly common need, but I've does several deep searches and, based on the dearth of info I've found on the subject, maybe it's not.
I was contacted about an opportunity to do VoiceXML--I don't have any experience with it, but apparently that might be okay. However, I did some initial research on the technology and it looks like it's pretty old, never gained much traction, etc. but I can't figure out what technolog(y, ies) replaced it. Is VoiceXML a good skill to acquire right now or is it dying out/never really caught on that much? If it is dying out, what is replacing it? Why?
VoiceXML is far from dead for developing Interactive Voice Response (IVR) systems. All of the top IVR platforms support VoiceXML because it is an open standards supported by the W3C and is still actively being worked on by the Voice Browser Working Group, along with many other associated standards. There is a good discussion on the progression of VoiceXML and development environments in this blog post.
The other advantage to VoiceXML is that it uses web based technologies for developing voice applications so you can leverage any skills you have in this area for development. But if you are new to IVR development there are other skills you will need to learn to be successful such as Voice User Interface (VUI) design, how to develop grammars for speech applications, and tuning speech applications. You would need to learn these skills regardless of the language or development environment.
Another approach for developing IVR applications that reduces the need for knowing VoiceXML is a framework like the open source project VoiceModel. If you already have skills in ASP.NET VoiceModel allows you to develop IVR application that run on VoiceXML platforms without understanding VoiceXML. It abstracts it to a higher level where you are just piecing together .NET objects to define your voice application. The other nice thing about VoiceModel is you can develop your IVR application once and it will run on a VoiceXML compatible IVR or a cloud-based IVR like Tropo. There are plans to also include support for Twilio.
And maybe that is the buzz you are hearing about VoiceXML dying. There are cloud-based IVR/telephony platforms that are cropping up such as Tropo and Twilio that do not use VoiceXML and use their own proprietary API's. To me this a step backwards towards the old days of IVR development where each IVR vendor had their proprietary development environment and languages. I am a firm believer in open standards like VoiceXML but I have to admit this new breed of IVR vendors are definitely making some traction in the market. I think it is too early to determine whether they will survive or not.
So the main question was; are there alternative to VoiceXML. The answer is yes. Two alternatives are the proprietary API's of cloud-based platforms of Tropo and Twilio. Another alternative is to use something like VoiceModel that abstracts away the need to know VoiceXML but lets you run the application on VoiceXML compatible platforms. And even another alternative is to use a development environment like VoiceObjects which also abstracts away much need to know VoiceXML.
Another question was; is VoiceXML a good skill to acquire. I would say yes, if you are developing a lot of IVR applications and especially if you are supporting existing ones. There are a lot of VoiceXML applications out there that require maintenance and currently the predominant IVR platform in the enterprise is VoiceXML compatible. If you are developing a one-off application you may want to look into something like Tropo or Twilio. With the caveat that if these start-ups go under you will be re-writing the application probably in VoiceXML. That is why I like opens standards. Or get the best of both worlds and look at something like VoiceModel.
VoiceXML is far from dying - most large commercial IVR deployments are based on VoiceXML and its cousin CCXML. There's significant traction, and VoiceXML developers are in demand. The next version of the VoiceXML spec is under development at the W3C.
While learning VoiceXML, you can try your skills out using the open source jVoiceXML (LGPL) http://jvoicexml.sourceforge.net/ or VoiceGlue (GPL) http://www.voiceglue.org/ or using the free developer hosting provided by Voxeo at http://evolution.voxeo.com/
That is one way to view VoiceXML, but it is much more complicated. Telephony technology is a niche market compared to anything like web development. However, within the telephony space, VoiceXML is the most common technology for IVR development and there isn't any other technology making inroads to replace it.
Part of the reason for slow growth of new technologies is related to the state of the market. From a vendor perspective, it is a tough market that is contracting. In the 80s and 90s, there are a lot of new startups. More than the market can support at a level that venture capitalists and other investors want. This has lead to consolidation (companies being bought up by other companies). Given the shortage of money, investment is low. That isn't to say there isn't any, but it is slow and very different than it was when VoiceXML first hit the scene.
Existing vendors usually provide a proprietary API stack, in a variety of languages, VoiceXML or both. It should be noted that even many of the platforms with VoiceXML support, provide development tools that hide the lower level APIs. These toolsets are often GUI based that came out of desires to have non-programmers write and manage IVRs.
The skill perspective attribute is a bit difficult to answer as its value is more related to your existing skill-set, marketability and future opportunities. In general, I wouldn't put a high value on VoiceXML knowledge because of the number of jobs is smaller than other mainstream skills. However, the typical implementations technologies are often similar to web based development (ASP/JSP/Servlet and the use of JavaScript), which are portable. As in any skill and job, it is what you make of it.
I've been programming IVR systems for about 20 years now, and I never bothered to learn VoiceXML. Here are the tools I use . . .
Visual Basic .NET
Dialogic voice PCI cards in Windows servers
CT ADE Visual Basic library from Syntellect http://www.syntellect.com/pages/products/ct_ade_eng.aspx
This setup allows me to code IVR systems in Visual Basic by calling routines provided by Syntellect. It eliminates the need to dive into VoiceXML, which is very granular by comparison.
Using the Syntellect routines, you can . . .
Make a call
Answer a call
Play a voice file to the caller
Get digits from caller
Record voice files
Transfer a call
Using these basic functions, you can create most any IVR system. It was very easy for me to then capture the "prompt" parameters in a SQL database and query entire script files to play to the caller.
This method is just for phone. And it that's all you need, this method is much faster to get up and running. Syntellect also has VoiceXML interfaces and solutions to help with VoIP connectivity.
Syntellect provides examples for both Visual Basic and C#. I think they also have some for C++. Hope this helps.
Twilio has demonstrated that a large number of use cases can be covered by a more intuitive and simple DSL. It takes no time to learn TwiML, while the VoiceXML learning curve can be significant.
There is also Restcomm, which is an open source (Mobicents) based implementation of the Twilio APIs. It can be used as a hosted IVR engine or on a public cloud:
http://www.telestax.com/restcomm/
ttps://aws.amazon.com/marketplace/pp/B00FFHJ6SU
I agree with DaveHolly. I used Syntellect CT-ADE for five years and implemented a successful IVR for medical dictation plus sending and receiving fax documents, and SQL Server database access. If you know how to program in C# or VB.NET, and want to use the fantastic Microsoft Visual Studio and .NET as your platform, then stay with CT-ADE or Glorsoft Velocity. In my opinion, by using XML as a programming language, something it was never designed for, you are giving up fifty years of computer science advances, such as structured programming, object-oriented programming, modern languages such as C#, debugging tools, etc..
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I would like to know a list of the most common application/websites/solutions where Erlang is used, successfully or not.
Explaining why it is used into a specific solution instead of others programming languages would be very much appreciated, too.
Listing BAD Erlang case studies (cases in which Erlang is misused) it would be interesting, as well.
From Programming Erlang:
(source: google.com)
Many companies are using Erlang in their production systems:
• Amazon uses Erlang to implement SimpleDB, providing database services as a part
of the Amazon Elastic Compute Cloud (EC2).
• Yahoo! uses it in its social bookmarking service, Delicious, which has more than
5 million users and 150 million bookmarked URLs.
• Facebook uses Erlang to power the backend of its chat service, handling more than
100 million active users.
• WhatsApp uses Erlang to run messaging servers, achieving up to 2 million connected users per server.
• T-Mobile uses Erlang in its SMS and authentication systems.
• Motorola is using Erlang in call processing products in the public-safety industry.
• Ericsson uses Erlang in its support nodes, used in GPRS and 3G mobile networks
worldwide.
The most popular open source Erlang applications include the following:
• The 3D subdivision modeler Wings 3D, used to model and texture polygon
meshes.
• The Ejabberd system, which provides an Extensible Messaging and Presence Protocol
(XMPP) based instant messaging (IM) application server.
• The CouchDB “schema-less” document-oriented database, providing scalability
across multicore and multiserver clusters.
• The MochiWeb library that provides support for building lightweight HTTP servers.
It is used to power services such as MochiBot and MochiAds, which serve
dynamically generated content to millions of viewers daily.
• RabbitMQ, an AMQP messaging protocol implementation. AMQP is an emerging
standard for high-performance enterprise messaging.
ejabberd is one of the most well know erlang application and the one I learnt erlang with.
I think it's the one of most interesting project for learning erlang because it is really building on erlang's strength. (However some will argue that it's not OTP, but don't worry there's still a trove of great code inside...)
Why ?
An XMPP server (like ejabberd) can be seen as a high level router, routing messages between end users. Of course there are other features, but this is the most important aspect of an instant messaging server. It has to route many messages simultaneously, and handle many a lot of TCP/IP connections.
So we have 2 features:
handle many connections
route messages given some aspects of the message
These are examples where erlang shines.
handle many connections
It is very easy to build scalable non-blocking TCP/IP servers with erlang. In fact, it was designed to solve this problem.
And given it can spawn hundreds of thousand of processes (and not threads, it's a share-nothing approach, which is simpler to design), ejabberd is designed as a set of erlang processes (which can be distributed over several servers) :
client connection process
router process
chatroom process
server to server processes
All of them exchanging messages.
route messages given some aspects of the message
Another very lovable feature of erlang is pattern matching.
It is used throughout the language.
For instance, in the following :
access(moderator, _Config)-> rw;
access(participant, _Config)-> rw;
access(visitor, #config{type="public"})-> r;
access(visitor, #config{type="public_rw"})-> rw;
access(_User,_Config)-> none.
That's 5 different versions of the access function.
Erlang will select the most appropriate version given the arguments received. (Config is a structure of type #config which has a type attribute).
That means it is very easy and much clearer than chaining if/else or switch/case to make business rules.
To wrap up
Writing scalable servers, that's the whole point of erlang. Everything is designed it making this easy. On the two previous features, I'd add :
hot code upgrade
mnesia, distributed relational database (included in the base distribution)
mochiweb, on which most http erlang servers are built on
binary support (decoding and encoding binary protocol easy as ever)
a great community with great open source projects (ejabberd, couchdb but also webmachine, riak and a slew of library very easy to embed)
Fewer LOCs
There is also this article from Richard Jones. He rewrote an application from C++ to erlang: 75% fewer lines in erlang.
The list of most common applications for Erlang as been covered (CouchDb, ejabberd, RabbitMQ etc) but I would like to contribute the following.
The reason why it is used in these applications comes from the core strength of Erlang: managing application availability.
Erlang was built from ground up for the telco environment requiring that systems meet at least 5x9's availability (99.999% yearly up-time). This figure doesn't leave much room for downtime during a year! For this reason primarily, Erlang comes loaded with the following features (non-exhaustive):
Horizontal scalability (ability to distribute jobs across machine boundaries easily through seamless intra & inter machine communications). The built-in database (Mnesia) is also distributed by nature.
Vertical scalability (ability to distribute jobs across processing resources on the same machine): SMP is handled natively.
Code Hot-Swapping: the ability to update/upgrade code live during operations
Asynchronous: the real world is async so Erlang was built to account for this basic nature. One feature that contributes to this requirement: Erlang's "free" processes (>32000 can run concurrently).
Supervision: many different strategies for process supervision with restart strategies, thresholds etc. Helps recover from corner-cases/overloading more easily whilst still maintaining traces of the problems for later trouble-shooting, post-mortem analysis etc.
Resource Management: scheduling strategies, resource monitoring etc. Note that the default process scheduler operates with O(1) scaling.
Live debugging: the ability to "log" into live nodes at will helps trouble-shooting activities. Debugging can be undertaken live with full access to any process' running state. Also the built-in error reporting tools are very useful (but sometimes somewhat awkward to use).
Of course I could talk about its functional roots but this aspect is somewhat orthogonal to the main goal (high availability). The main component of the functional nature which contributes generously to the target goal is, IMO: "share nothing". This characteristic helps contain "side effects" and reduce the need for costly synchronization mechanisms.
I guess all these characteristics help extending a case for using Erlang in business critical applications.
One thing Erlang isn't really good at: processing big blocks of data.
We built a betting exchange (aka prediction market) using Erlang. We chose Erlang over some of the more traditional financial languages (C++, Java etc) because of the built-in concurrency. Markets function very similarly to telephony exchanges. Our CTO gave a talk on our use of Erlang at CTO talk.
We also use CouchDB and RabbitMQ as part of our stack.
Erlang comes from Ericsson, and is used within some of their telecoms systems.
Outside telecoms, CouchDb (a document-oriented database) is possibly the best known Erlang application so far.
Why Erlang ? From the overview (worth reading in full):
The document, view, security and
replication models, the special
purpose query language, the efficient
and robust disk layout and the
concurrent and reliable nature of the
Erlang platform are all carefully
integrated for a reliable and
efficient system.
I came across this is in the process of writing up a report: Erlang in Acoustic Ray Tracing.
It's an experience report on a research group's attempt to use Erlang for Acoustic Ray Tracing. They found that while it was easier to write the program, less buggy, etc. It scaled worse, and performed 10x slower than a comparable C program. So one spot where it may not be well suited is CPU intensive scenarios.
Do note though, that the people wrote the paper were in the stages of first learning Erlang, and may not have known the proper development procedures for CPU intensive Erlang.
Apparently, Yahoo used Erlang to make something it calls Harvester. Article about it here: http://www.ddj.com/architect/220600332
What is erlang good for?
http://beebole.com/en/blog/erlang/why-erlang/
http://www.aquabu.com/2008/2/15/erlang-pragmatic-studio-day-3-notes
http://www.reddit.com/r/programming/comments/9q0lr/erlang_and_highfrequency_trading/
(jerf's answer)
It's important to realize that Erlang's 4 parts: the language itself, the VMs(BEAM, hipe) standard libs (plus modules on github, CEAN, etc.) and development environment are being steadily updated / expanded/improved. For example, i remember reading that the floating point performance improved when Wings3d's author realized it needed to improve (I can't find a source for this). And this guy just wrote about it:
http://marian-dan.com/wordpress/?p=324
A couple years ago, Tim Bray's Wide Finder publicity and all the folks starting to do web app frameworks and HTTP servers lead (at least in part) to improved regex and binaries handling. And there's all the work integrating HiPE and SMP, the dialyzer project, multiple unit testing and build libs springing up, ..
So its sweet spot is expanding, The difficult thing is that the official docs can't keep up very well, and the mailing list and erlang blogosphere volume are growing quickly
We are using Erlang to provide the back-end muscle power for our really real-time browser-based multi-player game Pixza. We don't use Flash or any other third-party plugins, though the game is real-time multi-player. We use pure JS and COMET techniques instead. And Erlang supports the "really realtimeliness" of Pixza.
I'm working for wooga, a social game company and we use Erlang for some of our game backends (basically http apis for millions of daily users) and auxiliary services like ios push notification provider, payment etc.
I think it really shines in network related tasks and it makes it kind of straight forward to structure and implement simple and complex network services alike in it. Distribution, fault tolerance and performance are easy to achieve because Erlang already has some of the key ingredients built in and they are being used for a long time in critical production infrastructure. So its not like "the new hip technology thing 0.0.2 alpha".
I know that other game companies use Erlang as well. You should be able to find presentations on slideshare about that.
Erlang draws its strength from being a functional language with no shared memory. Hence IMO, Erlang won't be suitable for applications that require in place memory manipulations. Image editing for example.
I am a newbie to Erlang and am trying to make a switch to Erlang for our latest project. Since this is going to be a real-time chat (long polled) system for file sharing on the fly, I realized after a bit of digging around that Erlang would be the most appropriate choice, because of high concurrency, plus people also suggested to use Yaws since it can handle upto 50k parallel connections.
Although, it sounds awesome, but since I am a newbie (both to erlang and comet applications), I am unable to understand the right technology stack / architecture of how this would work. Also, because of relatively less documentation, I am unable to figure out how the individual pieces would fit together (web server, application layer, DB, message queue) for such an application. The application is going to run off a desktop client only (no web presence required), and so we need to build a REST api for the functionalities.
It would be great, if someone could point me in the right direction to proceed.
Thanks
Nitrogen has a very slick Comet feature built-in. It will work with the three most popular Erlang web servers, including the one you're already considering, YAWS.
Nitrogen doesn't do anything in particular about data storage. It's not one of those web frameworks that insists on managing the DB for you. You're free to use Mnesia or whatever else you like. If this bothers you, you might consider Erlyweb instead. It doesn't do Comet for you like Nitrogen does, but it's more of the manage-everything-for-me sort of web framework.
You could use:
ejabberd as the XMPP server
mnesia as the database
YAWS as the WEB server
Message Queue : you can implement that in Erlang or use an enterprise solution such as RabbitMQ
The all new Zotonic application may inspire you. It's a webapp running off mochiweb for HTTP service with webmachine for the REST API. And it's using good ol' PostgreSQL as database.
It has comet support implemented.