In my understanding, IOLib and usocket have almost same abstraction level.
IOLib uses OS-backend sockets, on the other hand usocket uses Lisp-runtime-backend socket.
I just wonder which is a better choice for particular use cases.
For example, a server which needs great concurrency, or a client which focuses on portability, etc.
I think, this blogpost answers your question.
To sum up, if you're writing a library, which should work on all platforms and implementations (with a reasonable definition of "all"), use usocket. For other use-cases on the Unix platform, IOLib is probably more versatile. For example, it supports Unix domain sockets, as well as non-blocking IO.
By the way, I had ported cl-redis from usocket to IOLib and back - the API is very similar, although slightly different.
The code for USOCKET is much smaller and simpler than IOLib including dependencies. IOLib uses CFFI bindings to Linux features which are not present in some *BSD for example.
All other things being equal, minimal source code is always preferable because it means less bugs, because it is easier to understand and hack. Simpler code is faster to debug, and deploys more easily.
Other than that, they both seem to work roughly the same : they both provide kqueue/select to handle multiple connections within a single thread. I'm not sure about more advanced functions, like passing a unix file descriptor in a socket.
I'd say if you only like Linux, go with IOLib or USOCKET, if you target Linux and/or *BSD, or other commercial operating systems, or like to Keep It Simple & Stupid, go with USOCKET.
If your main goal is portability, apparently usockets is better choice, since as it's stated on this page:
USOCKET is a networking portability layer for BSD-style sockets.
Related
I'm wondering if there is a good resource for FORTH implementations on recent SOCs.
I'm mostly interested in bare metal versions, something that can sit net to RTOS on an ESP32 or RISC-V for instance (so gforth might not be ideal).
And particularly, I'm looking at least at a version that can do networking (e.g. via WIFI, ideally via a source network stack implementation, which in RTOS might not be too hard)
Sadly, it seems pretty hard to tickle useful info out of Google – it mostly seems to think I can't spell fourth. Many results I do find seem outdated, and or seem very commercial; and it feels wrong to pay licensing fees for a network stack in the age of micropython.
This Raspberry Pi JonesFORTH O/S looks pretty promising.
I wouldn't mind doing a bit of porting.
Even if your question could be interpreted as opinion-based and might be looking for resources only, I am going to provide two links to living and active maintained projects which might fit your requirements.
I'm wondering if there is a good resource for FORTH implementations on recent SOCs. I'm mostly interested in bare metal versions
You may have a look into
AmForth
... for the Atmel AVR8 Atmega micro controller family and some variants of the TI MSP430. The RISC-V CPU (32bit) is currently beeing worked on.
and
Mecrisp, Stellaris
an implementation of a standalone native code Forth for MSP430 microcontrollers
or Quintus
a rewrite of classic Mecrisp-Stellaris with almost the same look-and-feel for RISC-V architecture, RV32I, RV32IM or RV32IMC flavour, and it includes support for MIPS M4K cores. FPGAs are conquered by using FemtoRV32 softcores.
With both projects it is possible to achieve progress quite fast and easy depending what you try to achieve.
... can do networking (e.g. via WIFI, ideally via a source network stack implementation ... I wouldn't mind doing a bit of porting.
For a more detailed answer and more guidance this part would need more information and some clearance.
I can easily launch two or more separate Lua programs (running on LuaJIT2) on the same machine. But is there any way how these programs can communicate?
The best solution I can come up with is that each program can write a text file that orher program can read. But this is ree-aalllyyy slow solution (even if text files are saved on virtual RAM disks).
I know that thera are zeromq and other things that may help with this... The problem is that instructions are just too long and complicated / confusing (for me at least).
Any recommendations? Especially some working code example would be appreciated, no matter how simple. Even how to pass value of one variable from one LuaJIT2 process to another.
(I'm using Windows XP SP3, if that matters...)
Essentially, they keyword you should be looking for is "IPC" (Inter-Process Communication).
Some of the options you might want to explore:
Shared files
Shared memory
Network Sockets
Pipes (on POSIX systems)
Middleware utilizing one of the above
I can't really say that one of them is the best. The choice would depend on the other factors (needed speed, latency, what do you want to communicate), that you didn't provide. I just hope I pointed you in the right direction.
Also, if ZeroMQ is too complicated, i highly recommend you more reading, rather than looking for even simpler code examples. Lua is "do-it-yourself" language.
If you aren't really stick to Lua, take a look at PyRo (Python Remote Objects). Or you might want to implement something like this yourself.
What is an example of a situation where CORBA would be used? Is it just a matter of using an interface language (e.g. Java) to 'talk' to all applications?
CORBA might be used to build a language-independent, O/S-independent distributed system. For example, C++ on Linux developers could build a common distributed system with Java on Windows developers. IDL describes the interfaces that bind the two implementations over a common substrate (CORBA).
CORBA is also useful when building a plain old distributed object system - it has a rich set of services defined and is generally very well thought out. However, these days - depending on the language - many folks have opted for either simpler (e.g., RMI, protocol buffers) or message-based protocols (e.g., HTTP) for building distributed systems, so it's not as common. CORBA suffered from design-by-committee (esp on things like security).
More info:
http://en.wikipedia.org/wiki/Common_Object_Request_Broker_Architecture
You will see a list of real-life example of CORBA projects from below website.
http://www.cs.wustl.edu/~schmidt/TAO-users.html
TAO is one of the most popular C++ CORBA implementation available today. The project is pretty active.
CORBA technology vendors killed each other through incompatible and bureaucratic implementations. Today, you can safely consider CORBA to be a legacy technology; that is, use it if you have to deal with components that already expose themselves through COBA. Otherwise, stick to modern RPC/distribution standards like SOAP, or, better yet, REST/JSON.
Sorry. To answer your question: CORBA was intended to be what SOAP, REST, and others are today. Real-life examples of applications of the latter are examples of things attempted with the former.
I am a bit confused about this. If you're building a distributed application, which in some cases may perform parallel operations (although not necessarily mathematical), should you use ASIO or something like MPI? I take it MPI is a higher level than ASIO, but it's not clear where in the stack one would begin.
I know nothing about ASIO but from a quick Google it looks to me to be a lot lower level than MPI. For me the whole point of MPI is so that I can program against a higher level of abstraction from the messaging than, it seems, ASIO provides. Where you begin depends on your needs. For mine, parallelising scientific codes for high-performance, the obvious answer is MPI. I'm not sure I'd use it, or at least not sure it would be my default choice, if I were writing more general-purpose distributed, as opposed to parallel, applications. Well, actually, it probably would be my default choice to avoid learning another approach (most of which are less portable and less long-lived than MPI) but I'll admit it might not be the best choice if starting from an equal footing.
As far as I know MPI is currently incapable of handling the situation, when the new distributed nodes want to join the already started group. The problems also may occur if one of the nodes goes offline.
MPI does not reveal any network related machinery that is underneath. Thus if you would ever need something on the lower level -- you're in trouble. If you on the other hand do not aticipate such a need, then you'll save yourself a lot of time using MPI.
We are planning to develop a datamining package for windows. The program core / calculation engine will be developed in F# with GUI stuff / DB bindings etc done in C# and F#.
However, we have not yet decided on the model implementations. Since we need high performance, we probably can't use managed code here (any objections here?). The question is, is it reasonable to develop the models in FORTRAN or should we stick to C (or maybe C++). We are looking into using OpenCL at some point for suitable models - it feels funny having to go from managed code -> FORTRAN -> C -> OpenCL invocation for these situations.
Any recommendations?
F# compiles to the CLR, which has a just-in-time compiler. It's a dialect of ML, which is strongly typed, allowing all of the nice optimisations that go with that type of architecture; this means you will probably get reasonable performance from F#. For comparison, you could also try porting your code to OCaml (IIRC this compiles to native code) and see if that makes a material difference.
If it really is too slow then see how far that scaling hardware will get you. With the performance available through a modern PC or server it seems unlikely that you would need to go to anything exotic unless you are working with truly brobdinagian data sets. Users with smaller data sets may well be OK on an ordinary PC.
Workstations give you perhaps an order of magnitude more capacity than a standard dekstop PC. A high-end workstation like a HP Z800 or XW9400 (similar kit is available from several other manufacturers) can take two 4 or 6 core CPU chips, tens of gigabytes of RAM (up to 192GB in some cases) and has various options for high-speed I/O like SAS disks, external disk arrays or SSDs. This type of hardware is expensive but may be cheaper than a large body of programmer time. Your existing desktop support infrastructure shouldn be able to this sort of kit. The most likely problem is compatibility issues running 32 bit software on a 64-bit O/S. In this case you have various options like VMs or KVM switches to work around the compatibility issues.
The next step up is a 4 or 8 socket server. Fairly ordinary wintel servers go up to 8 sockets (32-48 cores) and perhaps 512GB of RAM - without having to move off the Wintel platform. This gives you fairly wide range of options within your platform of choice before you have to go to anything exotic1.
Finally, if you can't make it run quickly in F#, validate the F# prototype and build a C implementation using the F# prototype as a control. If that's still not fast enough you've got problems.
If your application can be structured in a way that suits the platform then you could look at a more exotic platform. Depending on what will work with your application, you might be able to host it on a cluster, cloud provider or build the core engine on a GPU, Cell processor or FPGA. However, in doing this you're getting into (quite substantial) additional costs and exotic dependencies that might cause support issues. You will probably also have to bring a third-party consultant who knows how to program the platform.
After all that, the best advice is: suck it and see. If you're comfortable with F# you should be able to prototype your application fairly quickly. See how fast it runs and don't worry too much about performance until you have some clear indication that it really will be an issue. Remember, Knuth said that premature optimisation is the root of all evil about 97% of the time. Keep a weather eye out for issues and re-evaluate your strategy if you think performance really will cause trouble.
Edit: If you want to make a packaged application then you will probably be more performance-sensitive than otherwise. In this case performance will probably become an issue sooner than it would with a bespoke system. However, this doesn't affect the basic 'suck it and see' principle.
For example, at the risk of starting a game of buzzword bingo, if your application can be parallelized and made to work on a shared-nothing architecture you might see if one of the cloud server providers [ducks] could be induced to host it. An appropriate front-end could be built to run locally or through a browser. However, on this type of architecture the internet connection to the data source becomes a bottleneck. If you have large data sets then uploading these to the service provider becomes a problem. It may be quicker to process a large dataset locally than to upload it through an internet connection.
I would advise not to bother with optimizations yet. First try to get a working prototype, then find out where computation time is spent. You can probably move the biggest bottlenecks out into C or Fortran when and if needed -- then see how much difference it makes.
As they say, often 90% of the computation is spent in 10% of the code.