the custom_federated_algorithms_2 tutorial presents a local_train function using tff.federated_computation.
There a comment saying "while we could have implemented this logic entirely in TensorFlow, relying on tf.data.Dataset.reduce...":
regarding this comment:
I didn't manage to actually convert the code to using tf.data.Dataset.reduce seems non-trivial and the debug comments really don't help
I wonder what is the motivation of using federated_computation in cases like this, I looked all over the guides and real did find an explanation for what is going on here and when should we use it.
thank's!
Addressing these two in order:
It may not be trivial to adapt the code given directly to use tf.data.Dataset.reduce; that comment is intended to call out that the logic expressed here is also expressible using the dataset-reduce primitive, as effectively it only represents a local reduction, there are no communications across placements happening here.
There are at least two distinct purposes of this demonstration. One is to show that TFF as a language does not necessarily rely on the in-graph looping constructs of TensorFlow; another is to demonstrate the ability to "capture" values using the federated computation decorator. This could be used to natively capture something like learning rate decay in TFF, by evaluating a function of the round number and closing over it in the manner above, though there are other ways to implement similar functionality, as demonstrated here for example.
I personally find this pattern a little confusing; reading into the question behind the question a little, I agree that it is confusing to use a federated_computation decorator where there is no communication happening. When writing TFF, I generally express all my local computation in TensorFlow proper (usually in a functional manner), and let TFF handle the communication only. The purpose of the second tutorial is to show that TFF proper is actually much more flexible than indicated by restricting oneself to using the pattern just described.
Related
I'm currently reading the documentation of WebSharper. In the section about FRP, it states:
Functional Reactive Programming (FRP) typically provides an Event type for event streams and a Behavior type for time-varying values, together with useful combinators on those.
...
However, for now we decided to avoid implementing FRP. Instead, we focus on a subset of functionality, defining time-varying View values similar to Behaviors, but without support for real-time sampling. Event streams are left for the user to tackle using callbacks or third-party libraries. This is a vast simplification over FRP and is much easier to implement efficiently.
As weak pointers become available in JavaScirpt, this decision might be revised, especially in light of OCaml React success.
In the more immediate future, we intend to provide Concurrent ML combinators to better support dealing with event streams and improve composition of Components.
However, I'm not sure what exactly is the difference between "Event type" and "Behavior type" described here. I Googled some articles/tutorials, but they don't seem to be very explicit on this either.
I'm not sure what I am missing out by not having "Event" in WebSharper's implementation.
Sorry if this question sounds fundamental. I'm not familiar with concepts related to FRP.
--
EDIT: I think I found the answer to my doubt on what is amiss without event streams, at FRP - Event streams and Signals - what is lost in using just signals?. The main points are:
event streams allow for accumulative updates, while behaviors can only depend on the current value of the observed elements.
if event and behavior are both implemented, they allow for recursion within the system.
The distinction between events and behaviours is something that dates back to the first paper on Functional Reactive Animations (PDF), which explains the distinction quite nicely. The idea is that:
Behaviours represent values that change in time - for example, mouse X coordinate varies in time, but it always has some value.
Events represent discrete events in the system - they happen every now and then and can trigger some change, but do not always have a value. For example, mouse click can happen, but you cannot ask "what's the current value of click".
These are very nice as theoretical ideas, because you can do different things with behaviours and events and they nicely capture some intuition behind different kinds of things in reactive systems.
In practice though, it is quite tricky to implement - most representations of "behaviours" end up using sampling, so they behave much like discrete events (maybe because that's how computers work?) and so only a few systems actually follow the original strict distinction.
Actually sometimes you are required to write some custom code to achieve some functionality, there would be 2 possible approaches there:
make your implementation by joining already given methods of
Objective-c
write your custom code
At that point I am confused which code base is better(in performance), that can only be decided if I luckily find Time-Complexity of Objective-c methods. So is there any way to know about it?
There are a lot of methods and functions you can call in the SDKs for iOS (and other Apple platforms), so this question is perhaps excessively broad.
But a discussion of time complexity is usually about algorithmic complexity, so we can restrict our scope to those calls that are the building blocks of algorithms where we measure time as a function of input size — that is, things like collection operations rather than, say, UIApplication registerForRemoteNotifications.
By and large, however, Apple doesn't talk much about the computational complexity of the high-level data structures in Cocoa. This probably has something do with Cocoa's design goals being strongly in favor of encapsulation, with simple interfaces hiding powerful, dynamic, and possibly adaptable implementations. Examining CoreFoundation — the open source implementations underlying the central bits of Cocoa, like collections — bears this out. Here's a great writeup on how NSArray is sometimes O(1) and sometimes not.
There's certainly something to be said for a philosophy where you're not supposed to care about the complexity of the tools you're using — tell it what you want to do, not how you want it done, and let it optimize performance for you, because it can second-guess you better than you can second-guess yourself. It goes well with a philosophy of avoiding premature optimization.
On the other hand, there's also some sense to a philosophy of having basic building blocks of enforced, predictable complexity so that you can more easily plan the complexity of the algorithms you build from them. Just to show that Apple seems to do it both ways, it appears this is the philosophy of choice for the Swift standard library.
When writing proofs I noticed that Agda's auto proof search frequently wouldn't find solutions that seem obvious to me. Unfortunately coming up with a small example, that illustrates the problem seems to be hard, so I try to describe the most common patterns instead.
I forgot to add -m to the hole to make Agda look at the module scope. Can I make that flag the default? What downsides would that have?
Often the current hole can be filled by a parameter of the function I am about to implement. Even when adding -m, Agda will not consider function parameters or symbols introduced in let or where clauses though. Is there something wrong with simply trying all of them?
When viewing a goal, symbols introduced in let or where clauses are not even displayed. Why?
What other habits can make using auto more effective?
Agda's auto proof search is hardwired into the compiler. That makes it fast,
but limits the amount of customization you can do. One alternative approach
would be to implement a similar proof search procedure using Agda's
reflection mechanism. With the recent beefed up version of reflection using
the TC monad,
you no longer need to implement your own unification procedure.
Carlos
Tome's been working on reimplementing these ideas (check out his code
https://github.com/carlostome/AutoInAgda ). He's been working on several
versions that try to use information from the context, print debugging info,
etc. Hope this helps!
I'm looking for an alternative to JMegahal that is just as simple, and easy to use, but yields better results. I know JMegahal uses Markov chains to generate new strings, and I know that they're not necessarily the best. I was pointed towards Bayesian Network as the best conceptual solution to this problem, but I cannot find any libraries for Java that are easy to use at all. I saw WEKA, but it seemed bloated, and hard to follow. I also saw JavaBayes, but it was almost completely undocumented (their javadocs contained little to no information, and the variables were poorly named) and the library was blatantly written in C-style, making it stand out in Java.
You might want to consider extending JMegahal to filter the input sentences. Back in the mid-90s, Jason Hutchens had written a C version of this 4th-order Markov strings algorithm (it was probably used as inspiration for the JMegahal implementation actually). At that time, Jason has added filters to improve on the implementations (by replacing 'you' by 'I', etc...). By doing some basic string manipulation meant to change the subject from the speaker to the system, the output became a lot more coherent. I think the expanded program was called HeX.
Reference 1
Reference 2
I've asked a couple of questions around this subject recently, and I think I'm managing to narrow down what I need to do.
I am attempting to create some "metrics" (quotes because these should not be confused with metrics relating to the performance of the application; these are metrics that are generated based on application data) in a Rails app; essentially I would like to be able to use something similar to the following in my view:
#metric(#customer,'total_profit','01-01-2011','31-12-2011').result
This would give the total profit for the given customer for 2011.
I can, of course, create a metric model with a custom result method, but I am confused about the best way to go about creating the custom metrics (e.g. total_profit, total_revenue, etc.) in such a way that they are easily extensible so that custom metrics can be added on a per-user basis.
My initial thoughts were to attempt to store the formula for each custom metric in a structure with operand, operation and operation_type models, but this quickly got very messy and verbose, and was proving very hard to do in terms of adding each metric.
My thoughts now are that perhaps I could create a custom metrics helper method that would hold each of my metrics (thus I could just hard code each one, and pass variables to each method), but how extensible would this be? This option doesn't seem very rails-esque.
Can anyone suggest a better alternative for approaching this problem?
EDIT: The answer below is a good one in that it keeps things very simple - though i'm concerned it may be fraught with danger, as it uses eval (thus there is no prospect of ever using user code). Is there another option for doing this (my previous option where operands etc. were broken down into chunks used a combination of constantize and get_instance_variable - is there a way these could be used to make the execution of a string safer)?
This question was largely answered with some discussion here: Rails - Scalable calculation model.
For anyone who comes across this, the solution is essentially to ensure an operation always has two operands, but an operand can either be an attribute, or the result of a previous calculation (i.e. it can be a metric itself), and it is thus highly scalable. This avoids the need to eval anything, and thus avoids the potential security holes that this entails.