I'm looking at moving a program that currently embeds a Python interpreter to use Lua. With Python it's fairly easy to use modulefinder, compileall, and zipfile to make a nice tidy zip containing all the external libraries used.
Does Lua have the ability to bundle up its libraries like that, or is there some better best practice for distributing programs that embed Lua?
As is typical with Lua, there's no one standard and a lot of people roll their own.
There's an effort to standardize on a package-management system called Lua Rocks, but I'm not sure how much momentum is behind it or how mature it is. (In 2008 it was not quite ready for prime time, but things may have changed.)
I myself am very low tech: if I want to distribute something, I just turn my Lua sources into C files and link them in with the binary. Finding and converting all the modules could be a bit tedious for me, but quite the easiest thing for my users—they don't even need to know that Lua is involved. I've posted a copy of my lua2c script at Pastebin. I have the option of compiling but I generally don't compile because the results are not portable and because the Lua compiler is so fast anyway.
It would be nice to have something more automated. I think it would probably take several days to write and debug a good tool.
People on the Lua mailing list will surely know more.
If it is pure Lua, you might also consider using squish
It's a tool that packs all Lua source files into a single file, with options to gzip/minify it.
Related
Hopefully this question won't be too convoluted or vague. I know what I want in my head, so fingers crossed I can get this across in text.
I'm looking for a language with a syntax of my own specification, so I assume I will need to create one myself. I've spent the last few days reading about compilers, lexers, parsers, assembly language, virtual machines, etc, and I'm struggling to sort everything out in terms of what I need to accomplish my goals (file attached at the bottom with some specifications). Essentially, I'm deathly confused as to what tools specifically I will need to use to go forward.
A little background: the language made would hopefully be used to implement a multiplayer, text-based MUD server. Therefore, it needs easy inbuilt functionality for creating/maintaining client TCP/IP connections, non-blocking IO, database access via SQL or similar. I'm also interested in security insofar as I don't want code that is written for this language to be able to be stolen and used by the general public without specialist software. This probably means that it should compile to object code
So, what are my best options to create a language that fits these specifications
My conclusions are below. This is just my best educated guess, so please contest me if you think I'm heading in the wrong direction. I'm mostly only including this to see how very confused I am when the experts come to make comments.
For code security, I should want a language that compiles and is run in a virtual machine. If I do this, I'll have a hell of a lot of work to do, won't I? Write a virtual machine, assembler language on the lower-level, and then on the higher-level, code libraries to deal with IO, sockets, etc myself, rather than using existing modules?
I'm just plain confused.
I'm not sure if I'm making sense.
If anyone could settle my brain even a little bit, I'd sincerely appreciate it! Alternatively, if I'm way off course and there's a much easier way to do this, please let me know!
Designing a custom domain-specific programming language is the right approach to a problem. Actually, almost all the problems are better approached with DSLs. Terms you'd probably like to google are: domain specific languages and language-oriented programming.
Some would say that designing and implementing a compiler is a complicated task. It is not true at all. Implementing compilers is a trivial thing. There are hordes of high-quality compilers available, and all you need to do is to define a simple transform from your very own language into another, or into a combination of the other languages. You'd need a parser - it is not a big deal nowdays, with Antlr and tons of homebrew PEG-based parser generators around. You'd need something to define semantics of your language - modern functional programming langauges shines in this area, all you need is something with a support for ADTs and pattern matching. You'd need a target platform. There is a lot of possibilities: JVM and .NET, C, C++, LLVM, Common Lisp, Scheme, Python, and whatever else is made of text strings.
There are ready to use frameworks for building your own languages. Literally, any Common Lisp or Scheme implementation can be used as such a framework. LLVM has all the stuff you'd need too. .NET toolbox is ok - there is a lot of code generation options available. There are specialised frameworks like this one for building languages with complex semantics.
Choose any way you like. It is easy. Much easier than you can imagine.
Writing your own language and tool chain to solve what seems to be a standard problem sounds like the wrong way to go. You'll end up developing yet another language, not writing your MUD.
Many game developers take an approach of using scripting languages to describe their own game world, for example see: http://www.gamasutra.com/view/feature/1570/reflections_on_building_three_.php
Also see: https://stackoverflow.com/questions/356160/which-game-scripting-language-is-better-to-use-lua-or-python for using existing languages (Pythong and LUA) in this case for in-game scripting.
Since you don't know a lot about compilers and creating computer languages: Don't. There are about five people in the world who are good at it.
If you still want to try: Creating a good general purpose language takes at least 3 years. Full time. It's a huge undertaking.
So instead, you should try one of the existing languages which solves almost all of your problems already except maybe the "custom" part. But maybe the language does things better than you ever imagined and you don't need the "custom" part at all.
Here are two options:
Python, a beautiful scripting language. The VM will compile the language into byte code for you, no need to waste time with a compiler. The syntax is very flexible but since there is a good reason for everything in Python, it's not too flexible.
Java. With the new Xtext framework, you can create your own languages in a couple of minutes. That doesn't mean you can create a good language in a few minutes. Just a language.
Python comes with a lot of libraries but if you need anything else, the air gets thin, quickly. On a positive side, you can write a lot of good and solid code in a short time. One line of python is usually equal to 10 lines of Java.
Java doesn't come with a lot of frills but there a literally millions of frameworks out there which do everything you can image ... and a lot of things you can't.
That said: Why limit yourself to one language? With Jython, you can run Python source in the Java VM. So you can write the core (web server, SQL, etc) in Java and the flexible UI parts, the adventures and stuff, in Python.
If you really want to create your own little language, a simpler and often quicker solution is to look at tools like lex and yacc and similar systems (ANTLR is a popular alternative), and then you can generate code either to an existing virtual machine or make a simple one yourself.
Making it all yourself is a great learning-experience, and will help you understand what goes on behind the scenes in other virtual machines.
An excellent source for understanding programming language design and implementation concepts is Structure and Interpretation of Computer Programs from MIT Press. It's a great read for anyone wanting to design and implement a language, or anyone looking to generally become a better programmer.
From what I can understand from this, you want to know how to develop your own programming language.
If so, you can accomplish this by different methods. I just finished up my own a few minutes ago and I used HTML and Javascript (And DOM) to develop my very own. I used a lot of x.split and x.indexOf("code here")!=-1 to do so... I don't have much time to give an example, but if you use W3schools and search "indexOf" and "split" I am sure that you will find what you might need.
I would really like to show you what I did and past the code below, but I can't due to possible theft and claim of my work.
I am pretty much just here to say that you can make your own programming language using HTML and Javascript, so that you and other might not get their hopes too low.
I hope this helps with most things....
I am planning to implement a template engine to my erlang project, and the most important thing is the performance. Currently I have a lot of Velocity Java template, and I want to migrate templates available to erlang.
I googled it, and found things like;
http://www.ivan.fomichev.name/2008/05/erlang-template-engine-prototype.html
erlydtl
google-ctemplate
sgte
Pure erlang implementation would be the best, but c(c++) based template engine, i.e. google-ctemplate, performs better, I would use it with erlang linked in driver.
Have no experience on this matter yet, so anyone's suggestions would be super great.
thanks
My personal favourite is erlydtl. It compiles the template to an erlang module, so there's no filesystem access or parsing time consumed when you call 'render'.
I think rebar has erlydtl support these days, so getting your templates compiled is a lot less hassle than it used to be. Just name them *.dtl and they'll get built when you run rebar compile.
It should also be fairly competitive speed wise as it's in-process (skip the IPC cost of a port program), compiled (and could be compiled to native code if you wanted to), and generates iolists which are pretty efficient.
A common statement said regarding Lua is that it doesn't come with batteries included; meaning that it lacks a lot of extra libraries.
I think there are a lot of Lua libraries out there and more are being developed all the time, but it is likely people don't know about many of them since the Lua community in general is very pragmatic about getting work done and doesn't waste a lot of time with self promotion.
So what are some great Lua libraries that more people ought to know about?
Shameless self-promotion plug: http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/
I hope you find something that's useful there.
My personal favorites are:
LuaSocket, a socket library enabling the use of internet with Lua
The Kepler suite a set of libraries for web application development in Lua.
LuaSQL and LuaSQLite for toying with DB stuff.
All this apart (or not as matter a fact) I highly recommend murgaLua for a batteries-included-but-not-bloated Lua distribution. It's crossplatform, and packs (non exhaustive list):
a binding to FLTK for developing GUI applications
LuaSQLite for sql stuff
LuaSocket
Basic encyption with slncrypt (blowfish, sha1, ...)
Decent RNG
And since the last beta release even a binding to FANN
Audio via ProteAudio
FFI via alien
...
And this whole beast packs in a measly 782kB executable.
I do not think there is the lack of "self promotion" , Lua is one of the best "glue" languages out there (if not the best), therefore allot of the code written for Lua is application specific.
For example, I´ve written a pretty extensive (networking) utility library for Lua and a pretty decent IDE, but its product specific and wont be released for general use.
http://www.intellipool.se/idedoc/
If I write Java software for a BlackBerry, can it be decompiled using freely available tools similar to .NET Reflector, or is it a more elaborate process?
Software you write for any language, any platform, can be decompiled. It doesn't matter what code mangling (obfuscation) tools you use, it can be decompiled.
Any attempt to worry about this is going to be a waste of time. Just like DRM ;p
But the real point is, and I wish I had the link to the discussion I am thinking of because it was very good. But the point is this. Some one can decompile it, and if they just straight recompile it and try and resell it, what has been the point? It's still easy peasy to take them to court and win.
But you say "They can look at my code and figure out how I did it and redo it!". And to this I say: Don't flatter your self.
Think if you could get your hands on the source code to Windows. There would be a lot of "WTF are they doing here"? And "boy I would have done things differently". A few moments where you scratch your chin and go "Wow, nice". But over all, it's nothing you wouldn't have come to on your own. The real value is the time they spent to truly wrap their heads around the issue and come up with a solution.
Anyone who rips off your code won't be doing that. What is harder? writing new software or maintaining software? I think most developers would prefer the former.
So someone decompiles your software and either sells it in such an obvious way that you can easily prosecute, or they take the time to fully wrap their mind around the problems and design their own which in the end (years later?) will probably be completely different from yours.
It's just such a ridiculous scenario, I really wounder if anyone has ripped off a product by decompiling a competing product.
Don't worry about some one "stealing" your code. It CAN be done and there is nothing you can do to prevent it, but it won't be done, because it's ridiculous.
RIM's RAPC tool transforms standard J2ME .jar files into .cod files - this is a proprietary format, and no details of it have been published. What's known is that the .cod file sizes are smaller than .jar file sizes, and a lot of stuff gets stripped out (anything not reachable from a static context, basically).
Note that when .cod files get big enough (or more accurately when their code or data sizes get big enough) then RAPC splits the .cod into 2 or more separate cods (named CODNAME-2.cod, etc.), zips them and renames the .zip to .cod. You can unzip this, but you're still left with a bunch of .cod files that you can't decompile.
So generally, no you can't decompile BlackBerry application files. There's a tool called coddec which claims to be able to decompile .cod files, but I haven't had any luck with it.
Java code is easy to decompile. Search for obfuscators for Java Me, if you are interested in securing your code. Notably check this.
However, Adam's comment on why you shouldn't care should also be taken into account.
RIM's toolchain also runs your code through an optimizer (rapc) that makes it even hard to decompile than standard bytecode. Unless somebody spent a lot of time specifically in order to decompile it, it's pretty unlikely.
If you write a Midlet using pure J2ME and Sun's Compiler, the resulting bytecode can easily be decompiled. (Not that I have ever done this :P).
However if you use RIM's BlackBerry JDE and it's RAPC Compiler, your bytecode will be in a proprietary, undocumented format and it's really hard for somebody to decompile that or make any sense of your code.
Be aware though, its always possible to find out hard-coded Strings in an application, so you don't wanna hard code your TripeDES Key or something like that.
Pretty much all software can be decompiled. The question is only the amount of involvement and resources required to pull it off.
Yes, it can in theory, since it's essentially just Java bytecode. However, last time I did BB development (2 years ago) it was actually quite difficult to do in practice.
I like JAD for Java classes - http://www.kpdus.com/jad.html.
Echoing the above sentiment - don't worry about people stealing your code.
My two favorite things about decompiling:
You learn how the compiler treats your code. For example, you can verify that string concatenation is handled by the StringBuilder class. Or, you may uncover some thought-provoking "for" statements beyond the traditional "for (int i=0, i
You can really learn a product and extend it. I've used Microsoft OCS2007 Speech Server. I learned a lot about the product be decompiling some of their assemblies. I also extended some of the basic tools to fit my needs.
I am trying to build out a useful 3d game engine out of the Ogre3d rendering engine for mocking up some of the ideas i have come up with and have come to a bit of a crossroads. There are a number of scripting languages that are available and i was wondering if there were one or two that were vetted and had a proper following.
LUA and Squirrel seem to be the more vetted, but im open to any and all.
Optimally it would be best if there were a compiled form for the language for distribution and ease of loading.
One interesting option is stackless-python. This was used in the Eve-Online game.
The syntax is a matter of taste, Lua is like Javascript but with curly braces replaced with Pascal-like keywords. It has the nice syntactic feature that semicolons are never required but whitespace is still not significant, so you can even remove all line breaks and have it still work. As someone who started with C I'd say Python is the one with esoteric syntax compared to all the other languages.
LuaJIT is also around 10 times as fast as Python and the Lua interpreter is much much smaller (150kb or around 15k lines of C which you can actually read through and understand). You can let the user script your game without having to embed a massive language. On the other hand if you rip the parser part out of Lua it becomes even smaller.
The Python/C API manual is longer than the whole Lua manual (including the Lua/C API).
Another reason for Lua is the built-in support for coroutines (co-operative multitasking within the one OS thread). It allows one to have like 1000's of seemingly individual scripts running very fast alongside each other. Like one script per monster/weapon or so.
( Why do people write Lua in upper case so much on SO? It's "Lua" (see here). )
One more vote for Lua. Small, fast, easy to integrate, what's important for modern consoles - you can easily control its memory operations.
I'd go with Lua since writing bindings is extremely easy, the license is very friendly (MIT) and existing libraries also tend to be under said license. Scheme is also nice and easy to bind which is why it was chosen for the Gimp image editor for example. But Lua is simply great. World of Warcraft uses it, as a very high profile example. LuaJIT gives you native-compiled performance. It's less than an order of magnitude from pure C.
I wouldn't recommend LUA, it has a peculiar syntax so takes some time to get used to. Depending on who will be doing the scripting, this may not be a problem, but I would try to use something fairly accessible.
I would probably choose python. It normally compiles to bytecode, so you would need to embed the interpreter. However, if you must you can use PyPy to for example translate the code to C, and then compile it.
Embedding the interpreter is no issue. I am more interested in features and performance at this point in time. LUA and Squirrel are both interpreted, which is nice because one of the games i am building out is to include modifiable code, which has an editor in game.
I would love to hear about python, as i have seen its use within the battlefield series i believe.
python is also nice because it has actual OGRE bindings, just in case you need to modify something lower-level on the fly. I don't know of any equivalent bindings for Lua.
Since it's a C++ library, I would suggest either JavaScript or Squirrel, the latter being my personal favorite of the two for being even closer to C++, in particular to how it handles tables/structs and classes. It would be the easiest to get used to for a C++ coder because of all the similarities.
However, if you go with JavaScript and find an HTML5 version of Ogre3D, you should be able to port your game code directly into the web version with minimal (if any) changes necessary.
Both of these are a good choice, and both have their pros and cons, but both would definitely be the easiest to learn since you're likely already working in C++. If you're working with Java, the same may hold true, and if it's Game Maker, you wouldn't need either one unless you're trying to make an executable that people wouldn't need Game Maker itself to run, in which case, good luck finding an extension to run either of these.