The RxJava library used to have the parallel operator which was discontinued , however the current Spring Reactor project has a parallel operator
What stops RxJava from being able to implement the parallel operator ?
I can see that there is project called RxJavaParallel which is trying to do this but I can't wrap my head around why it's being handled in this manner ? Is there is a inherent design limitation with the current RxJava implementations which make it much harder ?
Update
The kind of problem I was hoping to get solved
Flowable.range(1, 100)
.parallel(10)
.runOn(Schedulers.io())
.map(v -> httpClient.blockingGet("http://server/item/" + v))
.sequential()
.observeOn(AndroidSchedulers.mainThread())
.subscribe(...);
This PR pointed by #akarnokd is exactly what I was looking for ! Wonder why it was not in there to begin with :)
Update
Thanks #akarnokd for the link and the reply, I guess it's this attitude that keeps the library simple for folks like myself. The extensions project is worth looking at in case you cannot find a feature you are looking for in the main RxJava project
Starting from 2.0.5, RxJava 2 now offers a similar parallel() API.
Related
I've recently started learning Elixir and I'm puzzled by the arbitrary-looking divide between things that are available as Elixir modules vs. direct calls to Erlang libs.
Particularly I've noticed that formatting a Float into a string/binary is best done (in recent versions of Elixir) by directly calling the following function:
:erlang.float_to_binary/2
Perusing older questions and forum posts, I've seen several Elixir functions doing the above job (and sometimes just calling the above function) that were obsoleted and removed from Elixir.
What's the system in deciding which functions get an Elixir wrapper/implementation vs. direct calls to Erlang libs?
One reason I'm confused is that 'rendering a float as a string' feels like an incredibly basic language function, so I feel that I'm missing something when I see that Elixir doesn't provide it 'natively'.
It seems Elixir’s Float.to_string/1 is better for this.
When writing object-oriented software, I use dependency injection a lot:
to compose together high-level functionality from lower-level capabilities: my account management service uses repositories and validation services rather than implementing them itself.
to isolate components from their dependencies: my account management service uses its dependencies through interfaces, so that I can swap implementations, mock for unit testing and so on.
What patterns exist in functional programming languages to achieve these goals?
edit: a commenter rightly asks: "what about just passing round functions?". I think that the following comment about function grouping hits the nail on the head - a service is a collection of functions with a shared set of dependencies that I can handle as an atomic group.
In Clojure it seems like protocols solve this nicely, but I was really wondering how the problem is solved more generally...
Some time ago I've read a post describing how dependency injection can be seen as currying in functional programming. I think it's very interesting, and it gives a good perspective on the topic.
At the small scale, things like currying and functions-as-parameters cut down the need for module dependencies. At a larger scale, things like Standard ML functors are very useful for this purpose. Racket has a system called units that does a good job on this too.
I developed a little library which I found helpful for DI in a functional-inspired (JavaScript) environment, it's nothing special, just a bit method I like.
I am currently working on an F# project that contains many parallel calculations. As being bound to the trimmed .Net 4 Silverlight Framework (because of the required Silverlight compatibility) I cannot use the available .Net implmenetations and may only use the Monitor object and simple locking by using the lock Keyword.
Do you have any idea how a Shared-Exclusive lock implementation for F# might be desigend best?
I did some functional programming before but haven't concentrated on doing that parallel stuff (yet).
I'm not quite sure what exactly you need - if you need standard mutual exclusion, then the lock function is available in the Silverlight version of F# runtime.
If you need something more complex (such as multiple readers, single writer), then you can rewrite your code to use F# agents and solve the problem more elegantly. If you can add more details about the higher-level structure of your code, then someone can post an example how to solve your particular problem.
Anyway, the following SO answer shows how to write a reusable agent for multiple readers/single writer:
Implement CCR Interleave Arbiter in F#
As mentioned in the comment, you should probably try to avoid writing locks and low-level synchronization primitives explicitly, as this is a source of infinite number of bugs. F# agents give you a higher-level abstraction that is easier to use.
Theres an excellent chapter on this in Expert F# 2.0, Chapter 13 Reactive, Asynchronous, and Parallel Programming.
See example 13.13 shows a nice Request gate, something similar may be of use.
By this I meant: when you design your app side effects free, etc, will F# code be automatically distributed across all cores?
No, I'm afraid not. Given that F# isn't a pure functional language (in the strictest sense), it would be rather difficult to do so I believe. The primary way to make good use of parallelism in F# is to use Async Workflows (mainly via the Async module I believe). The TPL (Task Parallel Library), which is being introduced with .NET 4.0, is going to fulfil a similar role in F# (though notably it can be used in all .NET languages equally well), though I can't say I'm sure exactly how it's going to integrate with the existing async framework. Perhaps Microsoft will simply advise the use of the TPL for everything, or maybe they will leave both as an option and one will eventually become the de facto standard...
Anyway, here are a few articles on asynchronous programming/workflows in F# to get you started.
http://blogs.msdn.com/dsyme/archive/2007/10/11/introducing-f-asynchronous-workflows.aspx
http://strangelights.com/blog/archive/2007/09/29/1597.aspx
http://www.infoq.com/articles/pickering-fsharp-async
F# does not make it automatic, it just makes it easy.
Yet another chance to link to Luca's PDC talk. Eight minutes starting at 52:20 are an awesome demo of F# async workflows. It rocks!
No, I'm pretty sure that it won't automatically parallelise for you. It would have to know that your code was side-effect free, which could be hard to prove, for one thing.
Of course, F# can make it easier to parallelise your code, particularly if you don't have any side effects... but that's a different matter.
Like the others mentioned, F# will not automatically scale across cores and will still require a framework such as the port of ParallelFX that Josh mentioned.
F# is commonly associated with potential for parallel processing because it defaults to objects being immutable, removing the need for locking for many scenarios.
On purity annotations: Code Contracts have a Pure attribute. I remember hearing the some parts of the BCL already use this. Potentially, this attribute could be used by parallellization frameworks as well, but I'm not aware of such work at this point. Also, I' not even sure how well code contacts are usable from within F#, so a lot of unknowns here.
Still, it will be interesting to see how all this stuff comes together.
No it will not. You must still explicitly marshal calls to other threads via one of the many mechanisms supported by F#.
My understanding is that it won't but Parallel Extensions is being modified to make it consumable by F#. Which won't make it automatically multi-thread it, should make it very easy to achieve.
Well, you have your answer, but I just wanted to add that I think this is the most significant limitation of F# stemming from the fact that it is a hybrid imperative/functional language.
I would like to see some extension to F# that declares a function to be pure. That is, it has no side-effects that are not denoted by the function's type. The idea would be that a function is pure only if it references other "known-pure" functions. Of course, this would only be useful if it were then possible to require that a delegate passed as a function parameter references a pure function.
I'd like to start a new network server project in a language that supports concurrency through fibers aka coroutines aka user-mode threads. Determining what exactly are my options has been exceedingly difficult as the term "coroutine" seems to be used quite loosely to mean a variety of things, and "fiber" is used almost exclusively in reference to the Win32 API.
For the purposes of this question, coroutines/fibers:
support methods that pause execution by yielding a result to the calling function from within a nested function (i.e. arbitrarily deep in the call stack from where the coroutine/fiber was invoked)
support transferring control to another arbitrary coroutine at its current point of execution (i.e. yield to a coroutine that did not call your coroutine)
What are my language options? I know Ruby 1.9 and Perl (Coro) both have support, what else? Anything with a mature gc and dynamic method invocation is sufficient.
greenlet extension meets your requirements in Python (regular one, not Stackless).
Greenlet API is a bit low-level, so I recommend using gevent that gives you API suitable for an application. (Disclaimer: I wrote gevent)
Lua supports coroutines, see http://lua-users.org/wiki/CoroutinesTutorial , give it a try!
Tcl 8.6, currently in beta, will support coroutines. For more info see the Tcl Wiki coroutine page
Stackless Python is another option that meets your requirements. If Python, Ruby and Perl are all unsuitable for your purposes (despite all meeting your stated requirements), you presumably have other unstated requirements or preferences -- care to spell them out?-)
Scheme has call-with-current-continuation which is a building block on which all kinds of flow control can be built. It definitely can support the two uses you mentioned.
There are many robust, widely available implementations of Scheme such as PLT Scheme and Chicken Scheme.