How might you call a Lisp program from a Rails application?... For example, allow the end user to enter a block of text in the Rails web app, have the text processed by the Lisp program and return results to the Rails app?
There are a couple ways that come to mind:
Execute the lisp program with Process. Communicate with the Lisp program via standard in, and have the Lisp program output its result over stdout.
Do the same thing as above, but communicate via named pipes instead. Have your Ruby code write data into a named pipe, then have the Lisp program read from that pipe, and write data out over another named pipe which you then read with your Ruby app. The Lisp program can either run in the background as a daemon that checks for data on its incoming pipe, or you can fire it up as-needed using Ruby's command-line utilities (as above).
Find a Ruby-Lisp bridge. I have no experience with such a bridge (nor do I know off-hand if one even exists) and I think the above 2 mechanisms are easier, but your mileage may vary.
Another simple way is to have Lisp running a HTTP server and contact Lisp from the outside via HTTP requests.
CL-JSON supports JSON-RPC. It's very easy to set up with a web server such as Hunchentoot to have a Lisp-based web service that anything that speaks JSON-RPC (e.g. this) can use.
It would depend on how often it's going to happen.
If it's once in a blue moon, then just run a backquote command that starts the lisp interpreter, or popen it and write to it.
If it happens all the time, you will need to have Lisp already running, so the question then is how to communicate. Any of the interprocessor mechanisms will work, but I would suggest a TCP socket for development, testing, and production flexibility.
If it happens a million times a day, but a toy Lisp would be good enough, it is a simple matter to implement Lisp with Ruby classes. This was done as chapter 8 of Practical Ruby Projects.
Related
NGINX and other servers offer the option to use named pipes (mkfifo).
Can erlang use these instead of ports for nif interaction. What if I wanted to make 70,000 connections to my NIF (don't judge).
In short, no.
This is covered in the Erlang FAQ on opening device files. It boils down to it being hard/impossible to write the Erlang runtime in a portable way across Unices (not to mention Windows) so that it can access things like device files and named pipes without blocking on at least some of them. That blocking would screw up the "soft realtime" nature of the Erlang runtime.
What is possible is to write a C program that communicates with the Erlang runtime as a "port process", and that program can communicate over the named pipe (and block or not or whatever without screwing up the Erlang runtime).
I have a Go program that starts a C/Lua process. Now I'd like to communicate between those two. For example, in the middle of the run of the child (the c/lua process), I'd like to ask the parent (the Go program) to do some calculations and wait for the result. I am not keen to use stdin/stdout for communication, as this is already used for regular output. Now I am thinking of using sockets for the communication, but I don't want to reinvent the wheel.
What are the obvious choices for this kind of communication?
Is there a (more or less) simple standard way to pass objects between Lua and Go? If not, text blobs would suffice.
Are Protocol Buffers suitable for this? Looks like overkill, but I have no experience here.
Besides all the usual IPC methods you've mentioned (yeah, a unix socket with protobuf should do it, and stdin/stdout as well), if you run the C/Lua code embedded in your program, and not start it as a process, you can actually communicate between the languages directly.
Using the cgo module, Go code can call C functions, and embedded C code can call Go functions. See here: http://golang.org/cmd/cgo/#hdr-C_references_to_Go
Also, you have a couple embeddable Lua binding libraries for Go which you can try, that let you call Lua code and let your Lua code call Go. see https://github.com/aarzilli/golua and https://github.com/stevedonovan/luar/
I now talk to the subprocess via regular tcp sockets. The child process (Lua) has luasocket built in and this seems to work on Windows, Mac and Linux without problems.
Also, I have (for now) defined my own very simple protocol which looks OK in the first steps.
Just in case anybody is interested: https://github.com/speedata/publisher/commit/ea253382c1096274bca2d4124867c39cd0d512e5 and the child commits implement the tcp connection.
I have a C++ code that implement a special protocol over the serial port. The code is multi-threaded and internally polls the serial port and do its own cyclic processing. I would like to call this driver from erlang and also receive events from this driver. My concern is that this C++ code is multi-threaded and also statefull meaning that when I call a certain function on the driver, it caches things internally which will be used/required on the subsequent calls of the driver. My questions are
1.Does NIF run in the same os process as the rest of my erlang proceses or NIF is launched in a separate os process?
2.Does it make sense to warp this multi-threaded stateful C++ code with NIF?
4.If NIF is not the right approach, what is the better way for me to make Elrang talk back and forth with this C++ code. I also prefer my C++ code to be inside the same OS process as the rest of my Erlang processes and as it looks like linked-in drivers are an option but not sure if the multi-threaded nature of my C++ code will be ok to that model. Plus I hear they can mess up elrang scheduler?
Unlike ports, NIFs are run within Erlang VM process, similar to drivers. Because of that, any NIF crashes will bring VM down as well. And, answering in advance, to your last question, NIFs, like drivers, may block your scheduler.
That depends on the functionality you are implementing by this C++ code. Due to the answer 1), you probably want to avoid concurrency in the C++ part, since it's a potential source of errors. It's not always possible, of course. But if you are implementing, say, some workers pool, go ahead and implement 1-threaded code, spawning it as many times as you need.
Drivers can be multi-threaded too, with same potential problems and quite similar performance (well, still slightly faster than NIFs). If you are not completely sure about your C++ code stability, use it as an Erlang port.
Speaking of the difference between NIFs and drivers, the former is synchronous natively, and the latter can be asynchronous (which can be really a huge advantage if you don't want to receive any answers for most of the commands). Drivers are easier to mess up and harder to implement (but once you grasp the main patterns and problems, they seem okay, actually).
Here's a good start for drivers:
http://www.erlang.org/doc/apps/erts/driver.html
And something similar (behold the difference in complexity) for NIFs:
http://www.erlang.org/doc/tutorial/nif.html
Good evening.
I'm looking for a method to share data from my application system-wide, so that other applications could read that data and then do whatever they want with it (e.g. format it for display, use it for logging, etc). The data needs to be updated dynamically in the method itself.
WMI came to mind first, but then you've got the issue of applications pausing while reading from WMI. Additionally, i've no real idea how to setup my own namespace or classes if that's even possible in Delphi.
Using files is another idea, but that could get disk heavy, and it's a real awful method to use for realtime data.
Using a driver would probably be the best option, but that's a little too intrusive on the users end for my liking, and i've no idea on where to even start with it.
WM_COPYDATA would be great, but i'm not sure if that's dynamic enough, and whether it'll be heavy on resources or not.
Using TCP/IP would be the best choice for over the network, but obviously is of little use when run on a single system with no networking requirement.
As you can see, i'm struggling to figure out where to go with this. I don't want to go into one method only to find that it's not gonna work out in the end. Essentially, something like a service, or background process, to record data and then allow other applications to read that data. I'm just unsure on methods. I'd prefer to NOT need elevation/UAC to do this, but if needs be, i'll settle for it.
I'm running in Delphi 2010 for this exercise.
Any ideas?
You want to create some Client-Server architecture, which is also called IPC.
Using WM_COPYDATA is a very good idea. I found out it is very fast, lightweight, and efficient on a local machine. And it can be broadcasted over the system, to all applications at once (to be used with care if some application does not handle it correctly).
You can also share some memory, using memory mapped files. This is may be the fastest IPC option around for huge amount of data, but synchronization is a bit complex (if you want to share more than one buffer at once).
Named pipes are a good candidates for local. They tend to be difficult to implement/configure over a network, due to security issues on modern Windows versions (and are using TCP/IP for network communication - so you should better use directly TCP/IP instead).
My personal advice is that you shall implement your data sharing with abstract classes, able to provide several implementations. You may use WM_COPYDATA first, then switch to named pipes, TCP/IP or HTTP in order to spread your application over a network.
For our Open Source Client-Server ORM, we implemented several protocols, including WM_COPY_DATA, named pipe, HTTP, or direct in-process access. You can take a look at the source code provided for implementation patterns. Here are some benchmarks, to give you data from real implementations:
Client server access:
- Http client keep alive: 3001 assertions passed
first in 7.87ms, done in 153.37ms i.e. 6520/s, average 153us
- Http client multi connect: 3001 assertions passed
first in 151us, done in 305.98ms i.e. 3268/s, average 305us
- Named pipe access: 3003 assertions passed
first in 78.67ms, done in 187.15ms i.e. 5343/s, average 187us
- Local window messages: 3002 assertions passed
first in 148us, done in 112.90ms i.e. 8857/s, average 112us
- Direct in process access: 3001 assertions passed
first in 44us, done in 41.69ms i.e. 23981/s, average 41us
Total failed: 0 / 15014 - Client server access PASSED
As you can see, fastest is direct access, then WM_COPY_DATA, then named pipes, then HTTP (i.e. TCP/IP). Message was around 5 KB of JSON data containing 113 rows, retrieved from server, then parsed on the client 100 times (yes, our framework is fast :) ). For huge blocks of data (like 4 MB), WM_COPY_DATA is slower than named pipes or HTTP-TCP/IP.
Where are several IPC (inter-process communication) methods in Windows. Your question is rather general, I can suggest memory-mapped files to store your shared data and message broadcasting via PostMessage to inform other application that the shared data changed.
If you don't mind running another process, you could use one of the NoSQL databases.
I'm pretty sure that a lot of them won't have Delphi drivers, but some of them have REST drivers and hence can be driven from pretty much anything.
Memcached is an easy way to share data between applications. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects).
A Delphi 2010 client for Memcached can be found on google code:
http://code.google.com/p/delphimemcache/
related question:
Are there any Caching Frameworks for Delphi?
Googling for 'delphi interprocess communication' will give you lots of pointers.
I suggest you take a look at http://madshi.net/, especially MadCodeHook (http://help.madshi.net/madCodeHook.htm)
I have good experience with the product.
I've grown to like erlang, and it's a great (cough) architectural fit to my problem. Meanwhile I still like to imagine that I can kludge erlang processes & asynchronous message passing in python (I am currently in therapy to rid myself of this obsession).
During a recent binge I came across 0MQ & I like its messaging features. These may be self-evident to an erlang/OTP expert, but I'm just a humble python programmer (my shrink will no doubt get to read this clever argument). The 0MQ user-guide states that it uses native OS threads, and not virtual "green" threads.
Is there a way to make 0MQ work with say eventlet/gevent?
Or, should I avoid the green-eyed monster and stick to a single Python app thread, with non-blocking I/O handled by 0MQ's message queuing & its own (skilled) use of native threads?
Or, check out of rehab & go back to erlang?
Responding to a stale thread because I am kind of in the same boat. Thought I would share my thoughts.
1: It looks like all the heavy lifting has already been done: https://github.com/traviscline/gevent-zeromq has integrated the gevent loop with a nonblocking zmq socket and even some Cpython speedups. It also seems to be (at the time of this writing), reasonably well maintainted.
2: It depends; if you are writing something that can use zmq without a ton of external event logic, then you should just use zmq. If OTOH you need to integrate with other protocols, you may want to use gevent (or twisted perhaps, although it has no workable zmq now at all). My projects generally require multiple protocols (ie: private queue manager, public http, public https, private memcache, etc), so I am investigating switching to gevent for quicker project turnaround than my current favorite: twisted.
3: You may want to skip zmq entirely and integrate with an existing erlang based solution like rabbitMQ; the performance advantages of zmq may not be as important as you think, and then you have an erlang message queue that easily integrates with python with existing libraries.
Also see: Messsage Queue comparison at second life wiki
Zero MQ now works with Eventlet:
https://lists.secondlife.com/pipermail/eventletdev/2010-October/000907.html