I have an application written in erlang, i added a supervisor for distribution and now after parsing the configFile.cfg in the supervisor, i want to pass the configuration to my old application.
i have something like this now:
-module(supervisor_sup).
start() ->
application_app:start().
what i want is:
-module(supervisor_sup).
-record(config,{param1,param2}).
%After parsing the configFile.cfg
Conf = #config{param1 = Param1,
param2 = Param2},
start(Conf) ->
application_app:start(Conf).
It is uncommon to start applications from supervisors or modules under supervisors. The preferred way is to use application dependency to make sure all applications are started, and in the right order.
However, if you want some configuration to be available from several different applications without having to parse the configuration more than once, maybe the gproc library is what you are looking for?
https://github.com/uwiger/gproc
gproc can be used to efficiently set global configuration and much more. Even in a distributed environment.
Related
What is the recommended way to have different values for application environment variables in an erlang application?
What I mean here is: how do you support different environment in your application (e.g. development, stage, production) in your erlang application? For example I would like tests using a specific fake service on a known host and production code use the real server on a different host.
You can use application config file as well. you can also pass the config as parameter while starting an erlang console that can help you in setting up environment variables. so you are pass test.config or production.config based on environment there by no need to compile the code and start them.
You can find more info here
http://www.erlang.org/doc/man/config.html
Dependency injection.
test_setup() -> [ {host,"http://..."}, ... ].
prod_setup() -> [ {host,"http://..."}, ... ].
test_start() -> start(test_setup()).
prod_start() -> start(prod_setup()).
start(Config) -> ... .
Alternately, policy modules. Make a policy whose interface matches the stuff you need, then pass in the name of the module containing the policy you want. Think ETS/DETS.
The main thing I'm confused about here (I think) is what the arguments to the qfun are supposed to be and what the return value should be. The README basically doesn't say anything about this and the example it gives throws away the second and third args.
Right now I'm only trying to understand the arguments and not using Riak for anything practical. Eventually I'll be trying to rebuild our (slow, MySQL-based) financial reporting system with it. So ignoring the pointlessness of my goal here, why does the following give me a badfun exception?
The data is just tuples (pairs) of Names and Ages, with the keys being the name. I'm not doing any conversion to JSON or such before inserting the data from the Erlang console.
Now with some {Name, Age} pairs stored in <<"people">> I want to use MapReduce (for no other reason than to understand "how") to get the values back out, unchanged in this first use.
riakc_pb_socket:mapred(
Pid, <<"people">>,
[{map, {qfun, fun(Obj, _, _) -> [Obj] end}, none, true}]).
This just gives me a badfun, however:
{error,<<"{\"phase\":0,\"error\":\"{badfun,#Fun<erl_eval.18.17052888>}\",\"input\":\"{ok,{r_object,<<\\\"people\\\">>,<<\\\"elaine\\\">"...>>}
How do I just pass the data through my map function unchanged? Is there any better documentation of the Erlang client than what is in the README? That README seems to assume you already know what the inputs are.
There are 2 Riak Erlang clients that serve different purposes.
The first one is the internal Riak client that is included in the riak_kv module (riak_client.erl and riak_object.erl). This can be used if you are attached to the Riak console or if you are writing a MapReduce function or a commit hook. As it is run from within a Riak node it works quite well with qfuns.
The other client is the official Riak client for Erlang that is used by external applications and connects to Riak through the protocol buffers interface. This is what you are using in your example above. As this connects through protocol buffers, it is usually recommended that MapReduce functions in Erlang are compiled and deployed on the nodes of the cluster as named functions. This will also make them accessible from other client libraries.
I think my code is actually correct and my problem lies in the fact I'm trying to use the shell to execute the code. I need to actually compile the code before it can be run in Riak. This is a limitation of the Erlang shell and the way it compiles funs.
After a few days of playing around, here's a neat trick that makes development easier. Exploit Erlang's RPC support and the fact it has runtime code loading, to distribute your code across all the Riak nodes:
%% Call this somewhere during your app's initialization routine.
%% Assumes you have a list of available Riak nodes in your app's env.
load_mapreduce_in_riak() ->
load_mapreduce_in_riak(application:get_env(app_name, riak_nodes, [])).
load_mapreduce_in_riak([]) ->
ok;
load_mapreduce_in_riak([{Node, Cookie}|Tail]) ->
erlang:set_cookie(Node, Cookie),
case net_adm:ping(Node) of
pong ->
{Mod, Bin, Path} = code:get_object_code(app_name_mapreduce),
rpc:call(Node, code, load_binary, [Mod, Path, Bin]);
pang ->
io:format("Riak node ~p down! (ping <-> pang)~n", [Node])
end,
load_mapreduce_in_riak(Tail).
Now you can refer to any of the functions in the module app_name_mapreduce and they'll be visible to the Riak cluster. The code can be removed again with code:delete/1, if needed.
What is the recommended way to load+reload fsx files? Just experimenting... yes yes right language right job ect ect..
I love how the following can be done in FSI:
#load "script.fsx";
open Script
> let p = script.x 1
Error: This expression was expected to have type string but here has int...
(* edit script.fsx x to make it int -> int *)
>
> #load "script.fsx"
> let p = script.x 1
val it : int = 2
But how do we do this for an application that we are running via fsi blah.fsx? Maybe something that is sitting in a while loop. It seems #load and #use must not be inside let or module.. i.e. you cannot use #load like let reload script = #load script, wonder why?
My original method was to have .fs files and recompile + relaunch each time I wanted to add/fix something. This method feels primitive.
Second method was to attempt to use the #load directive inside of a module, which turns out to not work (kind of makes sense in terms of scoping)...
module test1 =
#load #"C:\users\pc\Desktop\test.fsx"
open Test
module test2 =
...
Another way would be to create a new process for every module by loading fsi module.fsx with process diagnostics, but this seems horrible, inefficient and ugh.
I have a feeling deep in my heart that this will not be trivial inside .NET, but I would like to pose the question anyway, FSI does it... I wonder if I can leverage the FSI API or something (or at the least to copy their code)?
TL;DR I read the following about erlang and want it for myself in F#.
Erlang: Is there a way to reload changed modules into an already running node with rebar?
"...any time a module in your program changes on disk, the reloader will replace the running copy automatically."
I don't know if this would work in FS but in ML you can load a master file that loads all your files in your project and then executes any code that you need to use to knit them together and runs your application. To see an example of a massive app run from inside of a REPL look at the Isabelle/HOL site at the Cambridge laboratory of Computational Science http://www.cl.cam.ac.uk/research/hvg/Isabelle/installation.html. After downloading the app look in the src code directory for any file called root.ml. There will be half a dozen of them that control various levels of implementation. This is recursive because a top level file can call a file in several sub-directories that loads that particular sub-feature. This allows targeting your application to various scenarios depending on which top level file is executed.
Typical .NET Framework applications cannot unload/reload assemblies unless they are in an App Domains that are separate from the primary one that starts up with the application. This is essentially how most plugin systems are designed for applications that run on the full .NET Framework. Things may be changing post .NET Standard 2.0 in .NET Core with the Collectible Assemblies feature.
References:
https://github.com/dotnet/coreclr/issues/552
https://github.com/dotnet/corefx/issues/19773
A newbie question: I wrote my first rebar based erlang application. I want to configure some basic properites like server host etc. Where is the best place to put them and how should I load them into the app?
The next steps are to make a release and create a node in it. A node runs your application in a standalone Erlang VM. A good starting point for creating a release using rebar:
Erlang Application Management with Rebar
Once you have created a release. The configuration properties for all applications in your node can then be added to
{your-app}/{release}/files/sys.config
You can read individual properties as follows:
Val = application:get_env(APP, KEY)
Alternatively, all properties for your application can be read as
Config = application:get_all_env(APP)
In sys.config, properties can be added as a proplist.
Example:
{myapp,
[
{port, 1234},
{pool_size, 5}
]
}
I'm currently writing some functions that are related to lists that I could possibly be reused.
My question is:
Are there any conventions or best practices for organizing such functions?
To frame this question, I would ideally like to "extend" the existing lists module such that I'm calling my new function the following way: lists:my_funcion(). At the moment I have lists_extensions:my_function(). Is there anyway to do this?
I read about erlang packages and that they are essentially namespaces in Erlang. Is it possible to define a new namespace for Lists with new Lists functions?
Note that I'm not looking to fork and change the standard lists module, but to find a way to define new functions in a new module also called Lists, but avoid the consequent naming collisions by using some kind namespacing scheme.
Any advice or references would be appreciated.
Cheers.
To frame this question, I would ideally like to "extend" the existing lists module such that I'm calling my new function the following way: lists:my_funcion(). At the moment I have lists_extensions:my_function(). Is there anyway to do this?
No, so far as I know.
I read about erlang packages and that they are essentially namespaces in Erlang. Is it possible to define a new namespace for Lists with new Lists functions?
They are experimental and not generally used. You could have a module called lists in a different namespace, but you would have trouble calling functions from the standard module in this namespace.
I give you reasons why not to use lists:your_function() and instead use lists_extension:your_function():
Generally, the Erlang/OTP Design Guidelines state that each "Application" -- libraries are also an application -- contains modules. Now you can ask the system what application did introduce a specific module? This system would break when modules are fragmented.
However, I do understand why you would want a lists:your_function/N:
It's easier to use for the author of your_function, because he needs the your_function(...) a lot when working with []. When another Erlang programmer -- who knows the stdlb -- reads this code, he will not know what it does. This is confusing.
It looks more concise than lists_extension:your_function/N. That's a matter of taste.
I think this method would work on any distro:
You can make an application that automatically rewrites the core erlang modules of whichever distribution is running. Append your custom functions to the core modules and recompile them before compiling and running your own application that calls the custom functions. This doesn't require a custom distribution. Just some careful planning and use of the file tools and BIFs for compiling and loading.
* You want to make sure you don't append your functions every time. Once you rewrite the file, it will be permanent unless the user replaces the file later. Could use a check with module_info to confirm of your custom functions exist to decide if you need to run the extension writer.
Pseudo Example:
lists_funs() -> ["myFun() -> <<"things to do">>."].
extend_lists() ->
{ok, Io} = file:open(?LISTS_MODULE_PATH, [append]),
lists:foreach(fun(Fun) -> io:format(Io,"~s~n",[Fun]) end, lists_funs()),
file:close(Io),
c(?LISTS_MODULE_PATH).
* You may want to keep copies of the original modules to restore if the compiler fails that way you don't have to do anything heavy if you make a mistake in your list of functions and also use as source anytime you want to rewrite the module to extend it with more functions.
* You could use a list_extension module to keep all of the logic for your functions and just pass the functions to list in this function using funName(Args) -> lists_extension:funName(Args).
* You could also make an override system that searches for existing functions and rewrites them in a similar way but it is more complicated.
I'm sure there are plenty of ways to improve and optimize this method. I use something similar to update some of my own modules at runtime, so I don't see any reason it wouldn't work on core modules also.
i guess what you want to do is to have some of your functions accessible from the lists module. It is good that you would want to convert commonly used code into a library.
one way to do this is to test your functions well, and if their are fine, you copy the functions, paste them in the lists.erl module (WARNING: Ensure you do not overwrite existing functions, just paste at the end of the file). this file can be found in the path $ERLANG_INSTALLATION_FOLDER/lib/stdlib-{$VERSION}/src/lists.erl. Make sure that you add your functions among those exported in the lists module (in the -export([your_function/1,.....])), to make them accessible from other modules. Save the file.
Once you have done this, we need to recompile the lists module. You could use an EmakeFile. The contents of this file would be as follows:
{"src/*", [verbose,report,strict_record_tests,warn_obsolete_guard,{outdir, "ebin"}]}.
Copy that text into a file called EmakeFile. Put this file in the path: $ERLANG_INSTALLATION_FOLDER/lib/stdlib-{$VERSION}/EmakeFile.
Once this is done, go and open an erlang shell and let its pwd(), the current working directory be the path in which the EmakeFile is, i.e. $ERLANG_INSTALLATION_FOLDER/lib/stdlib-{$VERSION}/.
Call the function: make:all() in the shell and you will see that the module lists is recompiled. Close the shell.
Once you open a new erlang shell, and assuming you exported you functions in the lists module, they will be running the way you want, right in the lists module.
Erlang being open source allows us to add functionality, recompile and reload the libraries. This should do what you want, success.