I have wondered if is possible (and advisable!) to implement CSP on top of F# Agents. I think if F# already have it and work well, then maybe is possible to just provide a API to mimic CSP with channels, ALT and similars...
The main trouble is that Agents are async, and CSP block. Or how implement CSP in F#?
P.D: I have found https://github.com/Hopac/Hopac but wish to know how implement it, for learning and to avoid dependencies if possible.
P.D 2: I have found a elixir sample at http://blog.plataformatec.com.br/2014/10/playing-with-elixir-and-go-concurrency-models/ and erlang https://gist.github.com/kachayev/5426175.
The Wikipedia article on Communicating Sequential Processes (CSP) has a nice section detailing key differences between CSP and the Actor Model (F# Agents are based on the Actor Model of concurrency). The two differences that stand out that you'd absolutely need to address are synchronous vs. asynchronous communication and writing to channels vs. writing directly to the process.
It may be possible, but it looks to be difficult. On the most fundamental level CSP requires completely synchronous communication between processes while the F# Agent (MailboxProcessor) is asynchronous, so you'd have to build a system which would force synchronous communication between F# Agents. A possible solution might be to use the PostAndReply function.
The next major difference (and maybe the hardest to overcome): CSP writes to a specific channel while the F# actor model you write messages to a specific actor. In other words, if you have two processes A and B: with F# Agents A will send a message by explicitly saying B.post(<msg>), while in CSP A would write a message to a channel named chan and B would be told to explicitly read from the channel named chan (note that in CSP the channels of communication are independent of the processes whereas in F# Agent model the channel of communication is identical to the receiving Agent). This seems like a much more difficult difference to overcome. Just throwing and idea out there (I have don't know if it would actually work well): one possibility might be to rethink what the F# Agent represents: instead of having the Agent act as the process have Agents that act as the CSP channel.
For anyone curious about CSP, Clojure's core.async is based on CSP and Brave Clojure has a pretty good tutorial that helps explain how CSP works.
Related
The feature widely advertised about Erlang is its ability to hot-swap code modules while the app is running and is shown as a unique killer-feature not available in other languages.
Here is a quote from Joe Armstrong's book:
Most servers execute a fixed program, and
if you want to modify the behavior of the server, you have to stop the server
and then restart it with the modified code.
Though, in context of web development, the majority of languages support so-called hot-swapping of the code even without calling it so. It's just updating code and publishing it on the web via git or a continuous integration solution.
I know Erlang has a lot of applications in domains other than web, so I am sure it makes sense in those cases.
But, does this feature have benefits in comparison to dynamic languages used for web, like Python, Ruby, JavaScript? What are the cases for web development where it outperforms popular web-oriented languages?
The hot code swaping in erlang offers more than the ability to upgrade the code (I won't make any comparison with python, ruby or javascript, I have a very limited knowledge of them):
You can decide, for each node, when you will load a new version of code
Then 2 versions of code will be present in the VM, all the running processes will use the old version of each module until a next fully qualified call to this module occurs (Mod:Func/arity)
if you are using OTP behaviors, the server (or fsm or gen_event) will be called first with their code_change call_back, receiving in the parameters, the old version of the module. So it is possible to check either or not it is possible to manage the upgrade, and to perform any necessary operation on the state Data, ETS, process synchronization... before really jumping into the new code.
if you are not using OTP behavior, it is still possible to receive the messages of the form {system, From, Req} and then call sys:handle_system_msg/6 which in turn will call the code_change call_back.
This feature is not targeted at web development just as Erlang itself was not created with web development specifically in mind.
One possible area where this feature outperforms model used in general dynamic languages used for web is precise control over the way code is upgraded.
code can be updated not only between calls, but also during call
you can provide explicit path of upgrade for state related to call
Currently I am looking for a "best practice url structure" for (BPM) process control. IMHO a process should not be controlled via a RESTful API.
Are there any standards or best practices for this?
Further explanations:
I start a new process instance of type "approval" for workitem "0815":
[PUT]http://server/process/approval/0815/start
I approve that process (yes, there can only be one such process for a workitem):
[PUT]http://server/process/approval/0815/approve
One more thing: everything is asynchronous here! So I get a 202 which means that the process handler will try to execute the command on the process.
Sorry if the answer is a bit disappointing but I believe there is no such thing as a best practice or standard for using REST in that context.
Given that BPM is not a standard (it is a methodology), this leaves the door open for an "unlimited" number of technical implementations for BPMS vendors.
The answer on how to use REST APIs to control you processes will most likely depend on vendor-specific APIs.
Modern BPMS such as Bonita BPM expose such APIs out of the box. If you have to implement your own APIs, you might want to consider looking at those.
Cheers,
I am trying to design an event driven system where the elements of the system communicate by generating events that are responded to by other components of the system. It is intended that the components be independent of each other - or as largely independent as I can make them. The system will initially be implemented on Windows 7, and is being written in Delphi. The generated events will be generated by the Delphi code. I understand how to implement a system of the type described on a single machine.
I wish to design the system so that it can readily be deployed on different machine architectures in particular with different components running on a distributed architecture, which may well be different to Windows 7. There is no requirement for the system ever to communicate with any systems external to itself.
I have tried investigating the architecture I need to consider and have looked at the questions mentioned below. These seem to point towards utilising named pipes as a mechanism for inter-hardware communications. As a result of these investigations I have sketched out the following to describe my system - the first part of the diagram is the system as I am developing it; the second part what I have deduced I would need for possible future implementations.
This leads to the following points:
Can you pass events via named pipes?
Is this an appropriate and sensible structure to tackle this problem?
Are there better alternatives?
What have I forgotten (at this level of granularity)?
How is event driven programming implemented?
How do I send a string from one instance of my Delphi program to another?
EDIT:
I had not given the points arising from "#I give crap answers" response sufficient consideration. My initial responses to his points are:
Synchronous v Asynchronous - mostly asynchronous
Events will always be in a FIFO queue.
Connection loss - is not terribly important - I can afford to deal with this non-rigourously.
Unbounded queues are a perfectly good way of dealing with events passed (if they can be) - there is no expectation of large volume of event generation.
For maximum deployment flexibility (operating-system independent), I recommend to take a look at popular open source message brokers which run on the Java platform. Using standard protocols. they integrate well with Delphi and other programming languages, can be used with web applications, and have a large installed user base and active community.
They are quite easy to install and configure in a few minutes, and free / commercial clients for Delphi are available.
Some examples are:
Apache ActiveMQ
OpenMQ
JBoss HornetQ
I also recommend the book "Enterprise Integration Patterns" by Martin Fowler as an overview and introduction, with many simple recipes to handle specific problems.
Note that I am a developer of commercial Delphi clients for enterprise messaging systems, such as xmlBlaster, RabbitMQ, Amazon Simple Queue Service and the three brokers mentioned above.
I can only answer for your point 4 here: You have not yet decided if an event is synchronous or asynchronous. In the async case, you have to decide what to do when messages arrive. Do you have a queue? How big is the queue? Can one grab arbitrary elements in the queue or is it strictly FIFO. What happens if a message is lost (somebody axes the network cable)?
In the sync variant, the advantage is that you got delivery guarantees, but then what do you do when connections are suddenly lost?
Connection loss is going to be a problem. The more machines you have, the greater is the chance that they will occur. Decide how you will handle that.
Another trouble may be what you do if you have a large event and several small. Is the order of transfer FIFO or smallest-first? Can events be reeordered? What are the assumptions here?
The aside is that I hack Erlang a lot. In Erlang all the event-handling is already solved but it also means a specific model is chosen for you (async, unbounded queues, no guaranteed delivery, but detection of connection loss).
I suggest to look at RabbitMQ, http://www.rabbitmq.com/. It has the server and client. Just need some wrapper codes in delphi and you are ready to build your business logic
Cheers
This is probably just an application for a message queue.
http://msdn.microsoft.com/en-us/library/ms632590(v=vs.85).aspx
I have some data that I need to share between multiple services on multiple machines. Stuffing the data into a database or shuffling it over http won't work in this situation and ideally the different pieces of software will need to communicate with each other directly (or through one central coordinator that can send and receive).
Is it recommended to create and implement a network protocol or use some tool to do the communication?
If I did go the route of creating a protocol myself, it wouldn't have to be very complex. Under 10 different message types, but it would have to be re-implemented in a few different languages for this project, and support unicode. I have read plenty (and done some) with handling sockets, but don't have much knowledge in handling a protocol I create. Are there any good resources on this?
There are also things like ICE and RPC that look intresting. The limit of my experience is using ICE and XMLRPC for a few days each. Is this the better route to go? If so what tools are out there?
Recently I've been using Google Protocol Buffers for encoding and shipping data between different machines running software written in different languages. It is quite easy to do, and takes away a lot of the hassle of designing a custom protocol.
Without knowing what technologies and platforms you are dealing with, it's difficult to give you a very specific answer - so I'll try to give you some general feedback.
If the system(s) you are wishing to connect span more than a single platform and/or technology you are probably better using an existing transport mechanism and protocol to maximize the chance your base platform will already have a library (or multiple) to interact over it. Also, integrating security and other features in a stack with known behaviors is more likely to be documented (with examples floating around). RPC (and ICE, though I've less familiarity with it) has some useful capabilities, but it also requires a lot of control over the environment and security can be convoluted (particularly if you are passing objects between different languages).
With regards to avoiding polling, this is a performance related issue; there are design patterns which can help you to handle such things - if you understand how you need the system to work (e.g. the observer pattern - kind of a dont-call-us-we'll-call-you approach). The network environment you are playing in will dictate which options are actually viable (e.g. a local LAN will have different considerations from something which runs over a WAN or the internet). Factors like firewall tunneling, VPN traversal, etc. should play part in your final selected technology profile.
The only other major consideration (that I can think of just now... ;-)) would be to consider the type of data you need to pass about. Is it just text, or do you need to stream binary objects? Would an encoding format (like XML or JSON or bJSON) do the trick? You mention "less than ten message types" as part of the question, but is that the only information which would ever need to be communicated by the system?
Either way, unless the overhead of existing protocols is unacceptable you're better of leveraging established work 99% of the time. Creativity is great - but commercial projects usually benefit from well-known behaviors, even if not the coolest or slickest (kind of the "as long as it works..." approach).
hth!
I am considering Erlang as a potential for my upcoming project. I need a "Highly scalable, highly reliable" (duh, what project doesn't?) web server to accept HTTP requests, but not really serve up HTML. We have thousands of distributed clients (other systems, not users) that will be submitting binary data to central cluster of servers for offline processing. Responses would be very short, success, fail, error code, minimal data. We want to use HTTP since it is our best chance of traversing firewalls.
Given this limited information about the project, can you provide any weaknesses that might pop up using a technology like Erlang? For instance, I understand Erlang's text processing capabilities might leave something to be desired.
You comments are appreciated.
Thanks.
This sounds like a perfect candidate for a language like Erlang. The scaling properties of the language are very good, but if you're worried about the data processing abilities, you shouldn't be. It's a very powerful language, with many libraries available for developers. It's an old language, and it's been heavily used/tested in the past, so everything you want to do has probably already been done to some degree.
Make sure you use erlang version R11B5 or newer! Earlier versions of erlang did not provide the ability to timeout tcp sends. This results in stalled or malicious clients being able to execute a DoS attack on your application by refusing to recv data you send them, thus locking up the sending process.
See issue OTP-6684 from R11B5's release notes.
With Erlang the scalability and reliability is there but from your project definition you don't outline what type of text processing you will need.
I think Erlang's main limitation might be finding experienced developers in your area. Do some research on the availability of Erlang architects and coders.
If you are going to teach yourself or have your developers learn it on the job keep in mind that it is a very different way of coding and that while the core documentation is good a lot of people do wish there were more examples. Of course the very active community easily makes up for that.
I understand Erlang's text processing
capabilities might leave something to
be desired.
The starling project already provides basic unicode support and there is a EEP (Erlang Enhancement Proposal) currently in draft, but going in to bring it into the mainstream of Erlang/OTP support.
I encountered some problems with Redis read performance from Erlang. Here is my question. I tend to think the reason is Erlang-written module, which has troubles while processing tons of strings during communication with Redis.