Add custom loss function in Torch - lua

What are the necessary steps to implement a custom loss function in Torch?
It seems like you have to write an implementation for updateOutput and updateGradInput.
Is that all? So then you basically create a new class:
local CustomCriterion, parent = torch.class('CustomCriterion','nn.Criterion')
and implement the following two functions:
function CustomCriterion:updateOutput(input, target)
function CustomCriterion:updateGradInput(input, target)
Is that correct, or is there more to be done?
Also, for the provided criterions these functions are implemented in C, but I suppose a Lua implementation will also work, albeit possibly a little slower?

I've implemented functions of the form (in pseudo-code)
--assuming input is partitioned in input_a,input_b
-- target is accordingly partitionend in target_a, target_b
f(input)=MSE(input_a,target_a)+ custom_sutff(input_b,target_b)
quite a few times just the way you describe it. So, as far as I know, I think the answer to both your questions is yes.
Basically nn/MSECriterion.lua and this seem to back this up.

Related

A more standard __attribute__((warning("msg"))

In my C++ library I have a function that is still there, 1) for debugging 2) for small operations.
The function is basically a very slow fallback of more efficient versions.
(think of a loop of individual assigments vs memcpy for example)>
For this reason I would like to emit a warning as soon the function is invoked instantiated directly or indirectly. Without a warning it is not easy to test if the function is being invoked instantiated, because the function might be called instantiated trough several layers of template code.
I found that GCC's __attribute__((warning("slow function!"))) does the job quite well.
template<class T>
__attribute__((warning("careful this fun is very slow, redesign your algorithm")))
void slow_function(T){...}
However it is not standard or compatible with clang.
Is there a better alternative for this kind of compile time warning?
It looks like there is a standard [[deprecated("msg")]] attribute that also does the job, the problem is that it is confusing because there is nothing deprecated about this function, it is there for convenience.
There is also, I found recently a #pragma poison that might be applicable here, but I don't understand how it is used, besides the function is actually a member function of a template class, the examples do not consider this case. https://www.fluentcpp.com/2018/09/04/function-poisoning-in-cpp/

Swift: Why use a function that doesn't return..?

I'm sure this question must have been asked 100s of times before, but, what are the most famous scenarios in which I use a function (with everything that come with it -parameters, logic inside the function-), then I wouldn't need to return anything..? Why is that function sitting inside my code, then..?
If I may put it differently, what benefit do I get from a function that does something -or somethings-, then doesn't provide me with some feedback or a result of its execution..?
Thank you everyone.
This mostly depends on the reason, which caused the programmer to encapsulate the code in an function/method. If it is an asynchronously executed method, for example, there wouldn't be a result an just the moment someone calls the method.
I believe my problem was trying to understand the purpose of functions in Swift from a procedural point of view. When I learned PL programming, I was taught that functions return values, and the ones that don't return values are called procedures. Now, 3 months into learning Swift, my view is way clearer as to how it's being used in Swift, and OOP.

How to swap functions (e.g.for tests) in pure functional programming

I'm trying to understand what is an FP-alternative to good old dependency injection from OOP.
Say I have the following app (pseudocode)
app() is where application starts. It allows user to register and list user posts (whatever). These two functions are composed out of several other functions (register does it step by step, imperatively, while list posts really composes them (at least this is how I understand function composition).
app()
registerUser(u)
validate(u)
persist(u)
callSaveToDB(u)
notify(u)
sendsEmail
listPosts(u)
postsToView(loadUserPosts(findUser(u)))
Now I'd like to test this stuff (registerUser and listPosts) and would like to have stubbed functions so that I don't call db etc - you know, usual testing stuff.
I know it's possible to pass functions to functions e.g
registerUser(validateFn, persistFn, notifyFn, u)
and have it partially applied so it looks like registerUser(u) with other functions closed over and so on. But it all needs to be done on app boot level as it was in OOP (wiring dependencies and bootstraping an app). It looks like manually doing this will take ages and tons of boilerplate code. Is there something obvious I'm missing there? Is there any other way of doing that?
EDIT:
I see having IO there is not a good example. So what if I have function composed of several other functions and one of them is really heavy (in terms of computations) and I'd like to swap it?
Simply - I'm looking for FP way of doing DI stuff.
The way to answer this is to drop the phrase "dependency injection" and think about it more fundamentally. Write down interfaces as types for each component. Implement functions that have those types. Replace them as needed. There's no magic, and language features like type classes make it easy for the compiler to ensure you can substitute methods in an interface.
The previous Haskell-specific answer, shows how to use Haskell types for the API: https://stackoverflow.com/a/14329487/83805

How do programmers practice code reuse

I've been a bad programmer because I am doing a copy and paste. An example is that everytime i connect to a database and retrieve a recordset, I will copy the previous code and edit, copy the code that sets the datagridview and edit. I am aware of the phrase code reuse, but I have not actually used it. How can i utilize code reuse so that I don't have to copy and paste the database code and the datagridview code.,
The essence of code reuse is to take a common operation and parameterize it so it can accept a variety of inputs.
Take humble printf, for example. Imagine if you did not have printf, and only had write, or something similar:
//convert theInt to a string and write it out.
char c[24];
itoa(theInt, c, 10);
puts(c);
Now this sucks to have to write every time, and is actually kind of buggy. So some smart programmer decided he was tired of this and wrote a better function, that in one fell swoop print stuff to stdout.
printf("%d", theInt);
You don't need to get as fancy as printf with it's variadic arguments and format string. Even just a simple routine such as:
void print_int(int theInt)
{
char c[24];
itoa(theInt, c, 10);
puts(c);
}
would do the trick nickely. This way, if you want to change print_int to always print to stderr you could update it to be:
void print_int(int theInt)
{
fprintf(stderr, "%d", theInt);
}
and all your integers would now magically be printed to standard error.
You could even then bundle that function and others you write up into a library, which is just a collection of code you can load in to your program.
Following the practice of code reuse is why you even have a database to connect to: someone created some code to store records on disk, reworked it until it was usable by others, and decided to call it a database.
Libraries do not magically appear. They are created by programmers to make their lives easier and to allow them to work faster.
Put the code into a routine and call the routine whenever you want that code to be executed.
Check out Martin Fowler's book on refactoring, or some of the numerous refactoring related internet resources (also on stackoverflow), to find out how you could improve code that has smells of duplication.
At first, create a library with reusable functions. They can be linked with different applications. It saves a lot of time and encourages reuse.
Also be sure the library is unit tested and documented. So it is very easy to find the right class/function/variable/constant.
Good rule of thumb is if you use same piece three times, and it's obviously possible to generalize it, than make it a procedure/function/library.
However, as I am getting older, and also more experienced as a professional developer, I am more inclined to see code reuse as not always the best idea, for two reasons:
It's difficult to anticipate future needs, so it's very hard to define APIs so you would really use them next time. It can cost you twice as much time - once you make it more general just so that second time you are going to rewrite it anyway. It seems to me that especially Java projects of late are prone to this, they seem to be always rewritten in the framework du jour, just to be more "easier to integrate" or whatever in the future.
In a larger organization (I am a member of one), if you have to rely on some external team (either in-house or 3rd party), you can have a problem. Your future then depends on their funding and their resources. So it can be a big burden to use foreign code or library. In a similar fashion, if you share a piece of code to some other team, they can then expect that you will maintain it.
Note however, these are more like business reasons, so in open source, it's almost invariably a good thing to be reusable.
to get code reuse you need to become a master of...
Giving things names that capture their essence. This is really really important
Making sure that it only does one thing. This is really comes back to the first point, if you can't name it by its essence, then often its doing too much.
Locating the thing somewhere logical. Again this comes back to being able to name things well and capturing its essence...
Grouping it with things that build on a central concept. Same as above, but said differntly :-)
The first thing to note is that by using copy-and-paste, you are reusing code - albeit not in the most efficient way.
You have recognised a situation where you have come up with a solution previously.
There are two main scopes that you need to be aware of when thinking about code reuse. Firstly, code reuse within a project and, secondly, code reuse between projects.
The fact that you have a piece of code that you can copy and paste within a project should be a cue that the piece of code that you're looking at is useful elsewhere. That is the time to make it into a function, and make it available within the project.
Ideally you should replace all occurrances of that code with your new function, so that it (a) reduces redundant code and (b) ensures that any bugs in that chunk of code only need to be fixed in one function instead of many.
The second scope, code reuse across projects, requires some more organisation to get the maximum benefit. This issue has been addressed in a couple of other SO questions eg. here and here.
A good start is to organise code that is likely to be reused across projects into source files that are as self-contained as possible. Minimise the amount of supporting, project specific, code that is required as this will make it easier to reuse entire files in a new project. This means minimising the use of project specific data-types, minimising the use project specific global variables, etc.
This may mean creating utility files that contain functions that you know are going to be useful in your environment. eg. Common database functions if you often develop projects that depend on databases.
I think the best way to answer your problem is that create a separate assembly for your important functions.. in this way you can create extension methods or modify the helper assemble itself.. think of this function..
ExportToExcel(List date, string filename)
this method can be use for your future excel export functions so why don't store it in your own helper assembly.. i this way you just add reference to these assemblies.
Depending on the size of the project can change the answer.
For a smaller project I would recommend setting up a DatabaseHelper class that does all your DB access. It would just be a wrapper around opening/closing connections and execution of the DB code. Then at a higher level you can just write the DBCommands that will be executed.
A similar technique could be used for a larger project, but would need some additional work, interfaces need to be added, DI, as well as abstracting out what you need to know about the database.
You might also try looking into ORM, DAAB, or over to the Patterns and Practices Group
As far as how to prevent the ole C&P? - Well as you write your code, you need to periodically review it, if you have similar blocks of code, that only vary by a parameter or two, that is always a good candidate for refactoring into its own method.
Now for my pseudo code example:
Function GetCustomer(ID) as Customer
Dim CMD as New DBCmd("SQL or Stored Proc")
CMD.Paramaters.Add("CustID",DBType,Length).Value = ID
Dim DHelper as New DatabaseHelper
DR = DHelper.GetReader(CMD)
Dim RtnCust as New Customer(Dx)
Return RtnCust
End Function
Class DataHelper
Public Function GetDataTable(cmd) as DataTable
Write the DB access code stuff here.
GetConnectionString
OpenConnection
Do DB Operation
Close Connection
End Function
Public Function GetDataReader(cmd) as DataReader
Public Function GetDataSet(cmd) as DataSet
... And So on ...
End Class
For the example you give, the appropriate solution is to write a function that takes as parameters whatever it is that you edit whenever you paste the block, then call that function with the appropriate data as parameters.
Try and get into the habit of using other people's functions and libraries.
You'll usually find that your particular problem has a well-tested, elegant solution.
Even if the solutions you find aren't a perfect fit, you'll probably gain a lot of insight into the problem by seeing how other people have tackled it.
I'll do this at two levels. First within a class or namespace, put that code piece that is reused in that scope in a separate method and make sure it is being called.
Second is something similar to the case that you are describing. That is a good candidate to be put in a library or a helper/utility class that can be reused more broadly.
It is important to evaluate everything that you are doing with an perspective whether it can be made available to others for reuse. This should be a fundamental approach to programming that most of us dont realize.
Note that anything that is to be reused needs to be documented in more detail. Its naming convention be distinct, all the parameters, return results and any constraints/limitations/pre-requisites that are needed should be clearly documented (in code or help files).
It depends somewhat on what programming language you're using. In most languages you can
Write a function, parameterize it to allow variations
Write a function object, with members to hold the varying data
Develop a hierarchy of (function object?) classes that implement even more complicated variations
In C++ you could also develop templates to generate the various functions or classes at compile time
Easy: whenever you catch yourself copy-pasting code, take it out immediately (i.e., don't do it after you've already CP'd code several times) into a new function.

Usage of inline closures / function delegates in Actionscript

Why are inline closures so rarely used in Actionscript? They are very powerful and I think quite readable. I hardly ever see anyone using them so maybe I'm just looking at the wrong code. Google uses them in their Google Maps API for Flash samples, but I think thats the only place I've seen them.
I favor them because you have access to local variables in the scope that defines them and you keep the logic in one method and dont end up with lots of functions for which you have to come up with a name.
Are there any catches of using them? Do they work pretty much the same way as in C#.
I actually only just discovered that AS3 supports them, and I'm quite annoyed becasue I had thought I read that they were deprecated in AS#. So I'm back to using them!
private function showPanel(index:int):void {
_timer = new Timer(1000, 1);
_timer.addEventListener(TimerEvent.TIMER, function(event:Event):void
{
// show the next panel
showPanel(index++);
});
The biggest gotcha to watch out for is that often 'this' is not defined in the inline closure. Sometimes you can set a 'this', but it's not always the right 'this' that you would have available to set, depending on how you're using them.
But I'd say most of the Flex code I've worked on has had inline closures rampantly throughout the code--since callbacks are the only way to get work done, and often you don't need the bring out a whole separate function.
Sometimes when the function nested is getting to be too much, I'll break it out slightly with Function variables in the function; this helps me organize a bit by giving labels to the functions but keeping some of the characteristics of inline closures (access to the local variables, for example).
Hope this helps.
One additional problem is that garbage collection is broken when it comes to closures (at least in Flash 9). The first instance of a given closure (from a lexical standpoint) will never be garbage collected - along with anything else referenced by the closure in the scope chain.
I found what originally made me NOT want to do this, but I had forgotten the details:
http://livedocs.adobe.com/flex/3/html/16_Event_handling_6.html#119539
(This is what Mitch mentioned - as far as the 'this' keyword being out of scope)
So thats Adobe's answer, however I am much more likely to need to refer to local variables than 'this'.
How do others interpret Adobe's recommendation ?

Resources