I am a newbie to Erlang and am trying to make a switch to Erlang for our latest project. Since this is going to be a real-time chat (long polled) system for file sharing on the fly, I realized after a bit of digging around that Erlang would be the most appropriate choice, because of high concurrency, plus people also suggested to use Yaws since it can handle upto 50k parallel connections.
Although, it sounds awesome, but since I am a newbie (both to erlang and comet applications), I am unable to understand the right technology stack / architecture of how this would work. Also, because of relatively less documentation, I am unable to figure out how the individual pieces would fit together (web server, application layer, DB, message queue) for such an application. The application is going to run off a desktop client only (no web presence required), and so we need to build a REST api for the functionalities.
It would be great, if someone could point me in the right direction to proceed.
Thanks
Nitrogen has a very slick Comet feature built-in. It will work with the three most popular Erlang web servers, including the one you're already considering, YAWS.
Nitrogen doesn't do anything in particular about data storage. It's not one of those web frameworks that insists on managing the DB for you. You're free to use Mnesia or whatever else you like. If this bothers you, you might consider Erlyweb instead. It doesn't do Comet for you like Nitrogen does, but it's more of the manage-everything-for-me sort of web framework.
You could use:
ejabberd as the XMPP server
mnesia as the database
YAWS as the WEB server
Message Queue : you can implement that in Erlang or use an enterprise solution such as RabbitMQ
The all new Zotonic application may inspire you. It's a webapp running off mochiweb for HTTP service with webmachine for the REST API. And it's using good ol' PostgreSQL as database.
It has comet support implemented.
Related
I am trying to develop an iOS application that stores and loads data to and from a server. The data needs of the client can be pretty much narrowed down to REST. MY question is, is REST something widely used for data/server driven iOS applications? is there a paradigm proven better or more suitable for iOS apps?
if REST is the way to go, what server environment would you choose? what server side Technology? PHP? Java? something else? We'd set up a test/dev environment at first, but eventually we are going to deploy on services like Amazon cloud or any other hosting/cloud service.
Any insight will be most welcome.
So are you writing the backend too?
Most projects that I've worked with in the past few years use REST. It's made little difference in the implementation of the iOS app (I think...). I'm more concerned about the type of data I'm consuming, which is usually JSON - and ensuring that it is as lean as possible. People writing web services should be concerned about REST.
I've also worked with projects this year that use different technologies on the backend. Java on one, and Ruby on Rails on the other.
I know another guy who uses PHP on one project, and Ruby on another.
If I was to do a backend, I'd probably use Java - solely because I know it. Not the best excuse to pick a technology - but as a full time iOS developer, I don't have time to learn something new that I won't use very often.
If you are going to deploy on a cloud based service, see what technologies they support. Maybe picking the common denominator would be a wise choice to keep your options open. Some languages tend to have more expensive server hosting costs. Java is usually more expensive than PHP to host, I guess due to the complexities of running shared java VM's instead of PHP interpreter.
I have a fairly complex windows service (written in .net 4) with several sub systems that run in parallel.
I have implemented pretty good logging throughout, but I'm feeling I'm needing more info about what each subsystem is currently doing. This would be very useful for times that I need to stop the service for upgrade/bug fixes.
It would be nice to have a gui app that will show me the status for each part of the application that I'm interested in. I've had some ideas for how I'm going to do this, but I'd like to hear some others' ideas as well.
I'm interested in a solution that would be easy to plop down in a future windows service and I'm not looking for anything very complex.
Are there any tools for this sort of thing?
Have you done this yourself?
What about interprocess communication?
Since Windows services can no longer interact with the user session, you'll need to have a separate application that does the interacting for you. Based on the details of your question, I think you understand this.
The big question is how to facilitate the communication between your Windows service and the application. There are all kinds of approaches - shared memory, socket, pipe, remoting, etc. What I have used successfully is WCF. If your UI is going to reside on the same machine as the service, use the NetNamedPipeBinding. If you ever need access from a remote machine, you can change to the NetTcpBinding. I've found this flow chart helpful in binding selection.
.
If you're looking for a more formal framework approach that just straight WCF, have a look at Juval Lowy's Publish-Subscribe WCF Framework, which is described in pretty good detail in this MSDN article. The code is available to look at via the article, or you can download the source and example from Lowy's website here. Go to the Downloads section, filter by the Discovery category, and you'll see it there.
In my free time, I'm currently working on a web app written with Rails, and planning on writing "thick" clients for the desktop and various mobile platforms (who doesn't ?).
I like the concept of Thrift for its multi-language support, and the concept of having one IDL file generating appropriate code for clients (DRY !)
I was wondering what would be the best way / architecture to integrate the Thrift server and Rails.
The only options that come to mind seems sup-optimal :
call the wepapp APIs from the Thrift server to return data to the thick clients
plug the thrift server to the DB of the Rails app and do its thing.
For obvious reasons, this seems overkill, redundant and not flexible.
Any suggestion ?
thanks !
I'm not sure if its overkill :) But I suggest if you want to explore this topic even more that you also look into this thread.
Just started reading the OTP chapter on the great Erlang book by Francesco Cesarini. Are most Erlang applications such as MochiWeb, Riak, RabbitMQ, Zotonic, ejabberd and CouchDB OTP applications?
CouchDB had lot of problems with that but the newest sources published by Cloudant show CouchDB in rebar, so it must be otp compliant.
Riak - the same, on rebar (btw. the same devs rebar and riak). Btw2. it is very nicely written app, good place to learn good practices.
Zotonic source code looks like mess a little bit. I can see there 'application' etc. but directory structure do not look like any proper OTP node. Btw. even Licence is not added on the top of all modules :?
ejabberd is full of sups and apps, but it has also some interesting;) design choices, so maybe do not learn Erlang on this example.
The Hibari database app is definitely OTP-based. The server is broken into several OTP apps, including a small one for managing config and logging (gmt), a big one for the server itself (gdss), a small one for native Erlang clients (gdss_client), and separate OTP apps for each of the server-side protocol handlers (e.g. JSON-RPC, UBF, EBF/BERT).
Sometimes a picture is worth at least a few hundred words. I've got some screen captures from the Erlang "appmon" (application monitor) app that shows the supervisor-and-worker process tree. Sorry, the protocol handler apps aren't shown in image #01, but they would be if I had had them running when I captured the image.
The link is here: OTP 'appmon' screen shots
-Scott
MochiWeb follows basic OTP Design Principles: it utilizes supervisor(3) behaviour to restart crashed processes (module mochiweb_sup.erl), and gen_server(3) behaviour. However it implements many routines to process data, such as ones found in mochinum.erl for faster floating point numbers serialization and mochijson[2].erl to process JSON etc.
The bad thing with mochiweb (I made it clear why here) IMHO is that it uses questionable and officially undocumented (since 2003!) technique of modules parameterizing (module mochiweb_request.erl, notice -module(mochiweb_request, [Socket, Method, RawPath, Version, Headers]). in the head of the file). The same applies to another Erlang HTTP-library misultin (misultin_req.erl).
Correct me if I'm wrong.
I've been checking out the Nitrogen Project which is supposed to be the most mature web development framework for Erlang.
Erlang, as a language, is extremely impressive. However, with regards to Nitrogen, what I am not too keen about is using Erlang's rather uncommon syntax (unless you're native in PROLOG) to build UIs.
What is your experience with it as opposed to other mainstream web frameworks such as Django or Rails?
I've done very little with Nitrogen so far, but I've been monitoring the mailing list for months, so I think I have something useful to say about it.
To your concern about the syntax of Erlang and the Nitrogen framework, I'd respond that that sounds like a pure case of unfamiliarity, rather than unsuitability. Objectively, HTML is not a beautiful language, and it has plenty of quirks. You're used to this now, so it doesn't seem so bad. Give Nitrogen/Erlang a chance and you may find that you get used to it soon enough, too.
To your question about comparison to other languages and frameworks, I'd say the biggest difference is that with Nitrogen, the entire web site is being served directly by the Erlang runtime. Ruby on Rails has such a mode, but it's intended only for testing. Many other frameworks don't even offer the option of running everything within a single long-running process.
Running the entire web application and its underlying infrastructure within a single long-running process has significant implications on how the site runs:
With Apache, each child gets killed off every N connections, where N=500 or so, and you can't say whether a given child will always handle all of a given client's requests. Because HTTP is stateless but web apps almost always require some client state, an Apache child must rebuild its view of client state as part of handling a new connection. By default, this means going back to disk for persistent data stored about that client. There are alternatives like memcached, but these aren't built into the core of a LAMP type stack. With Erlang, nothing is killed off periodically, and Erlang offers standard facilities like Mnesia which provide disk-backed in-memory DBs.
Incidentally, if you're familiar with nginx, it's built on the same principles as Erlang, and it's fast for the same reason. The main difference between nginx and an Erlang instance running a web server is that nginx isn't a programming environment, so it still has to delegate a lot of processing to outside code. That means it shares the same IPC and persistent state problems as Apache.
Because the runtime stays up continuously and is a fully-functional programming environment, you can probably build more parts of your system in Erlang than with a lashed-together LAMP type stack. This magnifies the above benefits. The various parts of your system can coordinate via message passing and Mnesia instead of heavyweight IPC and MySQL, and all the pieces stay up and running continually, leading to less time-consuming state reconstruction.
A dozen or so Apache children all accessing the persistent client state data store is a lock-based hairball. The frameworks all handle locking and such for you transparently, but what they can't hide is the time it takes to do all this correctly.
Erlang is an impure functional language, which implies but does not require data purity; it is also built with multiprocessing in mind, going clear down to the core of the runtime design. These two facts mean you're less likely to spend time waiting on locks in an Erlang based server than one naively built on one of the other frameworks. It is certainly possible to optimize away lock delays in the other systems, but is that really what you want to be doing? Do you want to be on the thousandth team that has to learn how to optimize its web stack after the service becomes popular, or would you rather leave it all up to the tooling so you can spend your time doing something no one else has done yet?
I, too, was once concerned about clunky Erlang syntax. I've built a couple of tools to alleviate its annoyances for everyday web programming, and perhaps you will find one or both of them helpful:
ErlyDTL is an Erlang implementation of the Django Template Language; it's not available in Nitrogen, but it is available in other frameworks, such as Zotonic, Erlang Web, BeepBeep, and Chicago Boss
Chicago Boss is a full-stack Erlang framework that does a lot of code generation so that you can access data fields with function calls instead of Erlang's rather verbose record syntax (e.g. Person:name() instead of Person#person.name)
Note that Nitrogen does not include a database layer, so it's not really comparable to Rails or Django. For a comprehensive comparison of the database-driven frameworks, check out my answer to this StackOverflow question:
https://stackoverflow.com/questions/1822518/current-state-of-erlang-web-development-frameworks-template-languages/2898271#2898271
I would check out Webmachine if I were you. It is quite simple, fast, and leaves the interface up to you.
Erlang Web should also be considered mature. It is an MVC framework, whereas Nitrogen is more event based. It's a matter of preference.
I haven't used the other tools mentioned here except Webmachine, which I think it's a wonderful tool, but it is not a web framework like the others. It is as HTTP processor, and is ideal for building a restful interfaces.
I would also suggest you give the Erlang syntax a chance. Erlang is one of my favourite languages to use.