I'd like to start a new network server project in a language that supports concurrency through fibers aka coroutines aka user-mode threads. Determining what exactly are my options has been exceedingly difficult as the term "coroutine" seems to be used quite loosely to mean a variety of things, and "fiber" is used almost exclusively in reference to the Win32 API.
For the purposes of this question, coroutines/fibers:
support methods that pause execution by yielding a result to the calling function from within a nested function (i.e. arbitrarily deep in the call stack from where the coroutine/fiber was invoked)
support transferring control to another arbitrary coroutine at its current point of execution (i.e. yield to a coroutine that did not call your coroutine)
What are my language options? I know Ruby 1.9 and Perl (Coro) both have support, what else? Anything with a mature gc and dynamic method invocation is sufficient.
greenlet extension meets your requirements in Python (regular one, not Stackless).
Greenlet API is a bit low-level, so I recommend using gevent that gives you API suitable for an application. (Disclaimer: I wrote gevent)
Lua supports coroutines, see http://lua-users.org/wiki/CoroutinesTutorial , give it a try!
Tcl 8.6, currently in beta, will support coroutines. For more info see the Tcl Wiki coroutine page
Stackless Python is another option that meets your requirements. If Python, Ruby and Perl are all unsuitable for your purposes (despite all meeting your stated requirements), you presumably have other unstated requirements or preferences -- care to spell them out?-)
Scheme has call-with-current-continuation which is a building block on which all kinds of flow control can be built. It definitely can support the two uses you mentioned.
There are many robust, widely available implementations of Scheme such as PLT Scheme and Chicken Scheme.
Related
I wanted to try out in Dart some algorithms and patterns from Functional Programming, but a lot of them rely heavily on recursion, which might incur in serious memory leaks without Tail Call Optimization (TCO), which isn't mandatory for when implementing a language.
Is there an official statement on this topic from the Dart team or something about it in the documentation? I could probably figure out if this is currently present in the language by using Dart's Dev Tools and Profiling, however this way I would never be able to know the Dart team's intentions with respect to the topic, hence the raison d'être of this question.
Dart does not support tail-call optimization. There are no current plans to add it.
The primary reason is that it's a feature that you need to rely on in order to use, otherwise you get hugely inefficient code that might overflow the stack, and since JavaScript currently does not support tail call optimization, the feature cannot be efficiently compiled to JavaScript.
I am currently working on an F# project that contains many parallel calculations. As being bound to the trimmed .Net 4 Silverlight Framework (because of the required Silverlight compatibility) I cannot use the available .Net implmenetations and may only use the Monitor object and simple locking by using the lock Keyword.
Do you have any idea how a Shared-Exclusive lock implementation for F# might be desigend best?
I did some functional programming before but haven't concentrated on doing that parallel stuff (yet).
I'm not quite sure what exactly you need - if you need standard mutual exclusion, then the lock function is available in the Silverlight version of F# runtime.
If you need something more complex (such as multiple readers, single writer), then you can rewrite your code to use F# agents and solve the problem more elegantly. If you can add more details about the higher-level structure of your code, then someone can post an example how to solve your particular problem.
Anyway, the following SO answer shows how to write a reusable agent for multiple readers/single writer:
Implement CCR Interleave Arbiter in F#
As mentioned in the comment, you should probably try to avoid writing locks and low-level synchronization primitives explicitly, as this is a source of infinite number of bugs. F# agents give you a higher-level abstraction that is easier to use.
Theres an excellent chapter on this in Expert F# 2.0, Chapter 13 Reactive, Asynchronous, and Parallel Programming.
See example 13.13 shows a nice Request gate, something similar may be of use.
What are some real-world projects done in concatenative languages like Forth, Factor, Joy, etc.?
factorcode.org, concatenative.org and tinyvid.tv are powered by Furnace, a Factor web server and framework.
PostScript is concatenative, and there's obviously a huge number of applications of PostScript. It's just not a general purpose programming language.
As Greg wrote, postscript is the mammoth example.
Concatenative languages pop up everywhere, quite naturally, because of the trivial nature of the language runtime. It's a favourite for many firmwares: I first encountered Forth "in the flesh" in the bootloader for a Sun Sparcstation. It powers the firmware for the OLPC.
Ocaml's parent, Caml was based on realising the semantics of functional programming as the Categorical Abstract Machine (the CAM in Caml).
Bibtex uses a concatenative language to compile style files.
There is the somewhat-obsolete but very cool Quartus Forth for Palm which allowed full compiled application development on the Palm device (Forth as a minimalist language works rather well in those circumstances). Their home page lists several Palm apps.
This FIG page has a list of mostly-embedded projects including a reference to the very cool use of Forth by NASA.
I met a guy at an Apple conference in Queensland back in about 1991 who had retailed a road planning application written in MacForth.
Christopher Diggins was talking about his Cat language being used inside Microsoft to help optimise compilers but I don't know if that went anywhere.
I suspect PowerMOPS (the successor to Neon) may elude the definition of concatenative because its big deal is adding object-orientation, which implies instances.
Take a look at FORTH Inc, They list several projects that they and their customers did, using their FORTH.
Eserv and nncron are written in SP-Forth.
Bitcoin protocol, and most of the other cryptocoins, uses pubkey scripts and signature scripts for validation of transactions:
Pubkey scripts and signature scripts combine secp256k1 pubkeys and signatures with conditional logic, creating a programable authorization mechanism.
These scripts are written in a concatenative language:
The script language is a Forth-like stack-based language deliberately designed to be stateless and not Turing complete. Statelessness ensures that once a transaction is added to the block chain, there is no condition which renders it permanently unspendable. Turing-incompleteness (specifically, a lack of loops or gotos) makes the script language less flexible and more predictable, greatly simplifying the security model.
Part of the firmware on Macs (at least in the older PowerPC models) was written in Forth.
See: Link
By this I meant: when you design your app side effects free, etc, will F# code be automatically distributed across all cores?
No, I'm afraid not. Given that F# isn't a pure functional language (in the strictest sense), it would be rather difficult to do so I believe. The primary way to make good use of parallelism in F# is to use Async Workflows (mainly via the Async module I believe). The TPL (Task Parallel Library), which is being introduced with .NET 4.0, is going to fulfil a similar role in F# (though notably it can be used in all .NET languages equally well), though I can't say I'm sure exactly how it's going to integrate with the existing async framework. Perhaps Microsoft will simply advise the use of the TPL for everything, or maybe they will leave both as an option and one will eventually become the de facto standard...
Anyway, here are a few articles on asynchronous programming/workflows in F# to get you started.
http://blogs.msdn.com/dsyme/archive/2007/10/11/introducing-f-asynchronous-workflows.aspx
http://strangelights.com/blog/archive/2007/09/29/1597.aspx
http://www.infoq.com/articles/pickering-fsharp-async
F# does not make it automatic, it just makes it easy.
Yet another chance to link to Luca's PDC talk. Eight minutes starting at 52:20 are an awesome demo of F# async workflows. It rocks!
No, I'm pretty sure that it won't automatically parallelise for you. It would have to know that your code was side-effect free, which could be hard to prove, for one thing.
Of course, F# can make it easier to parallelise your code, particularly if you don't have any side effects... but that's a different matter.
Like the others mentioned, F# will not automatically scale across cores and will still require a framework such as the port of ParallelFX that Josh mentioned.
F# is commonly associated with potential for parallel processing because it defaults to objects being immutable, removing the need for locking for many scenarios.
On purity annotations: Code Contracts have a Pure attribute. I remember hearing the some parts of the BCL already use this. Potentially, this attribute could be used by parallellization frameworks as well, but I'm not aware of such work at this point. Also, I' not even sure how well code contacts are usable from within F#, so a lot of unknowns here.
Still, it will be interesting to see how all this stuff comes together.
No it will not. You must still explicitly marshal calls to other threads via one of the many mechanisms supported by F#.
My understanding is that it won't but Parallel Extensions is being modified to make it consumable by F#. Which won't make it automatically multi-thread it, should make it very easy to achieve.
Well, you have your answer, but I just wanted to add that I think this is the most significant limitation of F# stemming from the fact that it is a hybrid imperative/functional language.
I would like to see some extension to F# that declares a function to be pure. That is, it has no side-effects that are not denoted by the function's type. The idea would be that a function is pure only if it references other "known-pure" functions. Of course, this would only be useful if it were then possible to require that a delegate passed as a function parameter references a pure function.
I am trying to build out a useful 3d game engine out of the Ogre3d rendering engine for mocking up some of the ideas i have come up with and have come to a bit of a crossroads. There are a number of scripting languages that are available and i was wondering if there were one or two that were vetted and had a proper following.
LUA and Squirrel seem to be the more vetted, but im open to any and all.
Optimally it would be best if there were a compiled form for the language for distribution and ease of loading.
One interesting option is stackless-python. This was used in the Eve-Online game.
The syntax is a matter of taste, Lua is like Javascript but with curly braces replaced with Pascal-like keywords. It has the nice syntactic feature that semicolons are never required but whitespace is still not significant, so you can even remove all line breaks and have it still work. As someone who started with C I'd say Python is the one with esoteric syntax compared to all the other languages.
LuaJIT is also around 10 times as fast as Python and the Lua interpreter is much much smaller (150kb or around 15k lines of C which you can actually read through and understand). You can let the user script your game without having to embed a massive language. On the other hand if you rip the parser part out of Lua it becomes even smaller.
The Python/C API manual is longer than the whole Lua manual (including the Lua/C API).
Another reason for Lua is the built-in support for coroutines (co-operative multitasking within the one OS thread). It allows one to have like 1000's of seemingly individual scripts running very fast alongside each other. Like one script per monster/weapon or so.
( Why do people write Lua in upper case so much on SO? It's "Lua" (see here). )
One more vote for Lua. Small, fast, easy to integrate, what's important for modern consoles - you can easily control its memory operations.
I'd go with Lua since writing bindings is extremely easy, the license is very friendly (MIT) and existing libraries also tend to be under said license. Scheme is also nice and easy to bind which is why it was chosen for the Gimp image editor for example. But Lua is simply great. World of Warcraft uses it, as a very high profile example. LuaJIT gives you native-compiled performance. It's less than an order of magnitude from pure C.
I wouldn't recommend LUA, it has a peculiar syntax so takes some time to get used to. Depending on who will be doing the scripting, this may not be a problem, but I would try to use something fairly accessible.
I would probably choose python. It normally compiles to bytecode, so you would need to embed the interpreter. However, if you must you can use PyPy to for example translate the code to C, and then compile it.
Embedding the interpreter is no issue. I am more interested in features and performance at this point in time. LUA and Squirrel are both interpreted, which is nice because one of the games i am building out is to include modifiable code, which has an editor in game.
I would love to hear about python, as i have seen its use within the battlefield series i believe.
python is also nice because it has actual OGRE bindings, just in case you need to modify something lower-level on the fly. I don't know of any equivalent bindings for Lua.
Since it's a C++ library, I would suggest either JavaScript or Squirrel, the latter being my personal favorite of the two for being even closer to C++, in particular to how it handles tables/structs and classes. It would be the easiest to get used to for a C++ coder because of all the similarities.
However, if you go with JavaScript and find an HTML5 version of Ogre3D, you should be able to port your game code directly into the web version with minimal (if any) changes necessary.
Both of these are a good choice, and both have their pros and cons, but both would definitely be the easiest to learn since you're likely already working in C++. If you're working with Java, the same may hold true, and if it's Game Maker, you wouldn't need either one unless you're trying to make an executable that people wouldn't need Game Maker itself to run, in which case, good luck finding an extension to run either of these.