Is there a MapReduce library for Lua?
(I obviously can't find one)
Penlight might be useful.
lua-mapreduce is a map-reduce implementation in lua.
Here is a wiki page: https://github.com/rohitjoshi/lua-mapreduce/wiki
Penlight looks good.
Underscore.lua is a Lua version of underscore.js, which has map/reduce.
https://github.com/pakozm/lua-mapreduce
It differs from https://github.com/rohitjoshi/lua-mapreduce in the basis of the communication between the processes. In order to allow fault tolerancy, and to reduce the communication protocol complexity, this implementation relies on mongoDB. So, all the data is stored at auxiliary mongoDB collections.
Related
Are there any Distributed Hash Table implementations in Erlang? I searched the web and found a number of research papers but I did not find a mature implementation.
Riak is quite mature and mostly implemented in Erlang. It's a bit more than just a DHT.
Redis is an incredibly fast implementation I have used with success before, and there is a great Erlang binding for it : http://streamhacker.com/2009/12/21/erldis-erlang-redis-client/
I've skimmed Programming in Lua, I've looked at the Lua Reference.
However, they both tells me this function does this, but not how.
When reading SICP, I got this feeling of: "ah, here's the computational model underlying scheme"; I'm trying to get the same sense concerning Lua -- i.e. a concise description of it's vm, a "how" rather than a "what".
Does anyone know of a good document (besides the C source) describing this?
You might want to read the No-Frills Intro to Lua 5(.1) VM Instructions (pick a link, click on the Docs tab, choose English -> Go).
I don't remember exactly where I've seen it, but I remember reading that Lua's authors specifically discourage end-users from getting into too much detail on the VM; I think they want it to be as much of an implementation detail as possible.
Besides already mentioned A No-Frills Introduction to Lua 5.1 VM Instructions, you may be interested in this excellent post by Mike Pall on how to read Lua source.
Also see related Lua-Users Wiki page.
See http://www.lua.org/source/5.1/lopcodes.h.html . The list starts at OP_MOVE.
The computational model underlying Lua is pretty much the same as the computational model underlying Scheme, except that the central data structure is not the cons cell; it's the mutable hash table. (At least until you get into metaprogramming with metatables.) Otherwise all the familiar stuff is there: nested first-class functions with mutable local variables (let-bound variables in Scheme), and so on.
It's not clear to me that you'd get much from a study of the VM. I did some hacking on the VM a while back and it's a lot like any other register-oriented VM, although maybe a bit cleaner. Only a handful of instructions are Lua-specific.
If you're curious about the metatables, the semantics is described clearly, if somewhat verbosely, in Section 2.8 of the reference manual for Lua 5.1. If you look at the VM code in src/lvm.c you'll see almost exactly that logic implemented in C (e.g., the internal Arith function). The VM instructions are specialized for the common cases, but it's all terribly straightforward; nothing clever is involved.
For years I've been wanting a more formal specification of Lua's computational model, but my tastes run more toward formal semantics...
I've found The Implementation of Lua 5.1 very useful for understanding what Lua is actually doing.
It explains the hashing techniques, garbage collection and some other bits and pieces.
Another great paper is The Implmentation of Lua 5.0, which describes design and motivations of various key systems in the VM. I found that reading it was a great way to parse and understand what I was seeing in the C code.
I am surprised you refer to the C source for the VM as this is protected by lua.org and the tecgraf/puc rio in Brazil specially as the language is used for real business and commercial applications in a number of countries. The paper about The Implementation of lua contains details about the VM in the most detail it is permitted to include but the structure of the VM is proprietary. It is worth noting that versions 5.0 and 5' were commissioned by IBM in Europe for use on customer mainframes and their register-based version have a VM which accepts the IBM defined format of intermediate instructions.
I just learnt that Erlang can remote load code and modules onto all instances of a cluster by using the "nl" command. Can any other languages do this?
Technically any of the lisp dialects could do it. Since 'code is data' in lisp, passing some code onto a different box and 'eval'-ing it would do the job. SLIME does this to some extent via remote repl using sockets.
You can write a ClassLoader in java similar to the codeloader in erlang. Java ClassLoaders have a lot of isolation, so it can be a bit more complicated (but you could do some nice things with this if you use it to your advantage rather than think of it as the enemy).
ClassLoaders are easy to write, but java doesn't ship with one that does the same kinds of things erlang does. Java also doesn't have the clustering tools erlang does, so it's not particularly surprising.
In theory pure functional languages should have such possibility but till this moment I've heard only about Erlang too.
None that I know, but it should be possible to implement it in dynamic languages such as Python, Perl or Lisp.
Which libraries do you guys use for generic data structures like linked list, binary tree etc.?
What are the most common, efficient libraries? Can you name some?
GDSL - The Generic Data Structures Library
Gnulib - The GNU Portability Library
GLib
SGLIB
In C....if you're on linux
http://library.gnome.org/devel/glib/2.22/
not sure about windows.....I don't use it :P
glib contains a lot of data structures.
I'd like to start a new network server project in a language that supports concurrency through fibers aka coroutines aka user-mode threads. Determining what exactly are my options has been exceedingly difficult as the term "coroutine" seems to be used quite loosely to mean a variety of things, and "fiber" is used almost exclusively in reference to the Win32 API.
For the purposes of this question, coroutines/fibers:
support methods that pause execution by yielding a result to the calling function from within a nested function (i.e. arbitrarily deep in the call stack from where the coroutine/fiber was invoked)
support transferring control to another arbitrary coroutine at its current point of execution (i.e. yield to a coroutine that did not call your coroutine)
What are my language options? I know Ruby 1.9 and Perl (Coro) both have support, what else? Anything with a mature gc and dynamic method invocation is sufficient.
greenlet extension meets your requirements in Python (regular one, not Stackless).
Greenlet API is a bit low-level, so I recommend using gevent that gives you API suitable for an application. (Disclaimer: I wrote gevent)
Lua supports coroutines, see http://lua-users.org/wiki/CoroutinesTutorial , give it a try!
Tcl 8.6, currently in beta, will support coroutines. For more info see the Tcl Wiki coroutine page
Stackless Python is another option that meets your requirements. If Python, Ruby and Perl are all unsuitable for your purposes (despite all meeting your stated requirements), you presumably have other unstated requirements or preferences -- care to spell them out?-)
Scheme has call-with-current-continuation which is a building block on which all kinds of flow control can be built. It definitely can support the two uses you mentioned.
There are many robust, widely available implementations of Scheme such as PLT Scheme and Chicken Scheme.