I embedded YAWS in my application at production environment, and I use the function yaws:start_embedded/4 to start YAWS.
Below is my code:
Id = "my_server",
GconfList = [{logdir, "./log"}, {id, Id}],
SconfList = [{docroot, Docroot},
{port, Port},
{listen, Listen},
{appmods, [
{"/rest", mod_rest, []},
{"/file", mod_file, []}
]}
],
yaws:start_embedded(Docroot, SconfList, GconfList, Id).
I'd like to add another appmod, e.g: {"/upload", mod_upload, []}
Is it possible to add appmods at runtime without restarting YAWS?
You can add appmods at runtime by first retrieving the current configuration, using it to create a new configuration containing your new appmods, and then setting the new configuration.
Call yaws_api:getconf/0 to get a 3-tuple {ok, GlobalConf, ServerConfs} where GlobalConf is the global Yaws configuration and ServerConfs is a list of lists of Yaws server configurations. The global conf is a record type named gconf, and the server conf is a record type named sconf; both of these record types are defined in the yaws.hrl header file.
Work through the server configurations to find the one containing the appmods you want to change. This is slightly tricky because you're dealing with a list of lists, and you need to keep the shape of the overall data structure unchanged.
Once you find the sconf, create a new sconf instance from it by adding your new appmod to its current list of appmods. Each element of the appmod list is a tuple consisting of a URL path for the appmod and the name of the appmod module. An appmod tuple can also optionally contain a third field consisting of a list of paths under the first path to be excluded; see the description of exclude_paths in the Yaws appmod documentation for more details.
Replace the existing sconf value in ServerConfs with your new value.
Call yaws_api:setconf/2 to set the new configuration, passing the existing GlobalConf as the first argument and the new ServerConfs containing your new sconf as the second argument.
The am_extend module below shows how to do this. It exports an add/1 function that takes a function that can identify and augment the appmods in the particular server you care about.
-module(am_extend).
-export([add/1]).
add(AppmodAdder) ->
{ok, GlobalConf, ServerConfs} = yaws_api:getconf(),
NewServerConfs = add_appmod(ServerConfs, AppmodAdder),
yaws_api:setconf(GlobalConf, NewServerConfs).
add_appmod(ServerConfs, AppmodAdder) ->
lists:foldl(fun(Val, Acc) ->
Acc ++ [AppmodAdder(A) || A <- Val]
end, [], ServerConfs).
An example of using this code is to pass the function below as the AppmodAdder argument for am_extend:add/1. For this example, we're looking for a server that has an appmod path "/sse" so we can add another appmod to that server for the path "/sse2". Any server conf we don't care about is just returned unchanged.
-include_lib("yaws/include/yaws.hrl").
add_sse2(#sconf{appmods=AM}=SC) ->
case lists:keyfind("/sse", 1, AM) of
false ->
SC;
_ ->
SC#sconf{appmods=[{"/sse2", my_sse_module}|AM]}
end.
Note that our add_sse2/1 function must be compiled with yaws.hrl included so it has the definition for the sconf record available.
Related
I am working on an Elixir/Phoenix app that is intended to create a user in my UserController controller. There is a /lib/helpers/user_helpers directory with a number of modules (each in a separate file). These modules all have a common namespace UserHelpers.ModuleName. In each of these modules I have a function called apply which I want to apply to the user data. For example if I have the following file structure:
-lib
-helpers
-user_helpers
-module1
-module2
-...
-moduleN-1
-moduleN
where each of module1 and module2 contains a function apply(user_info) which returns user_info. In my UserController I have the function create(conn, params) in which I want to run the following:
user_data
|> UserHelpers.Module1.create
|> UserHelpers.Module2.create
|> ...
|> UserHelpers.ModuleN-1.create
|> UserHelpers.ModuleN.create
But I'm unsure how to dynamically load all of the modules in the UserHelpers folders to do the above. Any suggestions?
Assuming, that your application is called :my_app and helpers have the .ex extension and/or explicitly compiled into your application:
with {:ok, list} <- :application.get_key(:my_app, :modules) do
list
|> Enum.filter(& &1 |> Module.split |> Enum.take(1) == ~w|UserHelpers|)
|> Enum.reduce(user_data, fn m, acc -> apply(m, :create, acc) end)
end
:application.get_key(:my_app, :modules) returns the list of modules, known to that application. The second line filters out those unneeded, and the latter one applies their :create functions to user_data subsequently.
You probably want to embed Enum.sort just before the last line to sort the modules to apply in the appropriate order.
How can I access yaws file without including it's extension? Say,
www.domain.com/listen.yaws => www.domain.com/listen
I could not find any specific documentation for this from yaws documentation/appmod.
I think the question is ultimately clarified!
You can find one example of how to accomplish this in the "Arg Rewrite" section (7.1.2) of the Yaws PDF documentation. Set the variable arg_rewrite_mod in your server configuration to the name of an Erlang module supporting rewriting:
arg_rewrite_mod = my_rewriter
To support rewriting, the my_rewriter module must define and export an arg_rewrite/1 function, taking an #arg{} record as its argument:
-module(my_rewriter).
-export([arg_rewrite/1]).
-include_lib("yaws/include/yaws_api.hrl").
rewrite_pages() ->
["/listen"].
arg_rewrite(Arg) ->
Req = Arg#arg.req,
{abs_path, Path} = Req#http_request.path,
case lists:member(Path, rewrite_pages()) of
true ->
Arg#arg{req = Req#http_request{path = {abs_path, Path++".yaws"}}};
false ->
Arg
end.
The code includes yaws_api.hrl to pick up the #arg{} record definition.
The rewrite_pages/0 function returns a list of pages that must be rewritten to include ".yaws" suffixes; in this example, it's just the /listen page you mention in your question. If in arg_rewrite/1 we find the requested page in that list, we append ".yaws" to the page name and include it in a new #arg{} we return to Yaws, which then continues dispatching the request based on the new #arg{}.
I have the following F# program that retrieves a webpage from the internet:
open System.Net
[<EntryPoint>]
let main argv =
let mutable pageData : byte[] = [| |]
let fullURI = "http://www.badaddress.xyz"
let wc = new WebClient()
try
pageData <- wc.DownloadData(fullURI)
()
with
| :? System.Net.WebException as err -> printfn "Web error: \n%s" err.Message
| exn -> printfn "Unknown exception:\n%s" exn.Message
0 // return an integer exit code
This works fine if the URI is valid and the machine has an internet connection and the web server responds properly etc. In an ideal functional programming world the results of a function would not depend on external variables not passed as arguments (side effects).
What I would like to know is what is the appropriate F# design pattern to deal with operations which might require the function to deal with recoverable external errors. For example if the website is down one might want to wait 5 minutes and try again. Should parameters like how many times to retry and delays between retries be passed explicitly or is it OK to embed these variables in the function?
In F#, when you want to handle recoverable errors you almost universally want to use the option or the Choice<_,_> type. In practice the only difference between them is that Choice allows you to return some information about the error while option does not. In other words, option is best when it doesn't matter how or why something failed (only that it did fail); Choice<_,_> is used when having information about how or why something failed is important. For example, you might want to write the error information to a log; or perhaps you want to handle an error situation differently based on why something failed -- a great use case for this is providing accurate error messages to help users diagnose a problem.
With that in mind, here's how I'd refactor your code to handle failures in a clean, functional style:
open System
open System.Net
/// Retrieves the content at the given URI.
let retrievePage (client : WebClient) (uri : Uri) =
// Preconditions
checkNonNull "uri" uri
if not <| uri.IsAbsoluteUri then
invalidArg "uri" "The URI must be an absolute URI."
try
// If the data is retrieved successfully, return it.
client.DownloadData uri
|> Choice1Of2
with
| :? System.Net.WebException as webExn ->
// Return the URI and WebException so they can be used to diagnose the problem.
Choice2Of2 (uri, webExn)
| _ ->
// Reraise any other exceptions -- we don't want to handle them here.
reraise ()
/// Retrieves the content at the given URI.
/// If a WebException is raised when retrieving the content, the request
/// will be retried up to a specified number of times.
let rec retrievePageRetry (retryWaitTime : TimeSpan) remainingRetries (client : WebClient) (uri : Uri) =
// Preconditions
checkNonNull "uri" uri
if not <| uri.IsAbsoluteUri then
invalidArg "uri" "The URI must be an absolute URI."
elif remainingRetries = 0u then
invalidArg "remainingRetries" "The number of retries must be greater than zero (0)."
// Try to retrieve the page.
match retrievePage client uri with
| Choice1Of2 _ as result ->
// Successfully retrieved the page. Return the result.
result
| Choice2Of2 _ as error ->
// Decrement the number of retries.
let retries = remainingRetries - 1u
// If there are no retries left, return the error along with the URI
// for diagnostic purposes; otherwise, wait a bit and try again.
if retries = 0u then error
else
// NOTE : If this is modified to use 'async', you MUST
// change this to use 'Async.Sleep' here instead!
System.Threading.Thread.Sleep retryWaitTime
// Try retrieving the page again.
retrievePageRetry retryWaitTime retries client uri
[<EntryPoint>]
let main argv =
/// WebClient used for retrieving content.
use wc = new WebClient ()
/// The amount of time to wait before re-attempting to fetch a page.
let retryWaitTime = TimeSpan.FromSeconds 2.0
/// The maximum number of times we'll try to fetch each page.
let maxPageRetries = 3u
/// The URI to fetch.
let fullURI = Uri ("http://www.badaddress.xyz", UriKind.Absolute)
// Fetch the page data.
match retrievePageRetry retryWaitTime maxPageRetries wc fullURI with
| Choice1Of2 pageData ->
printfn "Retrieved %u bytes from: %O" (Array.length pageData) fullURI
0 // Success
| Choice2Of2 (uri, error) ->
printfn "Unable to retrieve the content from: %O" uri
printfn "HTTP Status: (%i) %O" (int error.Status) error.Status
printfn "Message: %s" error.Message
1 // Failure
Basically, I split your code out into two functions, plus the original main:
One function that attempts to retrieve the content from a specified URI.
One function containing the logic for retrying attempts; this 'wraps' the first function which performs the actual requests.
The original main function now only handles 'settings' (which you could easily pull from an app.config or web.config) and printing the final results. In other words, it's oblivious to the retrying logic -- you could modify the single line of code with the match statement and use the non-retrying request function instead if you wanted.
If you want to pull content from multiple URIs AND wait for a significant amount of time (e.g., 5 minutes) between retries, you should modify the retrying logic to use a priority queue or something instead of using Thread.Sleep or Async.Sleep.
Shameless plug: my ExtCore library contains some things to make your life significantly easier when building something like this, especially if you want to make it all asynchronous. Most importantly, it provides an asyncChoice workflow and collections functions designed to work with it.
As for your question about passing in parameters (like the retry timeout and number of retries) -- I don't think there's a hard-and-fast rule for deciding whether to pass them in or hard-code them within the function. In most cases, I prefer to pass them in, though if you have more than a few parameters to pass in, you're better off creating a record to hold them all and passing that instead. Another approach I've used is to make the parameters option values, where the defaults are pulled from a configuration file (though you'll want to pull them from the file once and assign them to some private field to avoid re-parsing the configuration file each time your function is called); this makes it easy to modify the default values you've used in your code, but also gives you the flexibility of overriding them when necessary.
I'm developing an application with 1 supervisor and several workers. Each one of those workers will just open a tcp socket, execute a listen, and then accept connections, spawning a process for each client as they arrive (I dont mind supervising these).
I'd like to make the listening addresses configurable in the application configuration, so I can have as many addresses to listen as needed (at least 1 address is needed, but any number can be specified). Each address is an ip (or hostname) and a port address. So far, nothing new.
My question is about how to declare, start, and supervise an unknown number of workers. My solution was to dynamically generate (at runtime) the result of the init/1 function in the supervisor code, like this:
-define(
CHILD(Name, Args),
{Name, {
?MODULE, start_listen, Args
}, permanent, 5000, worker, [?MODULE]
}
).
init([]) ->
{ok, Addresses} = application:get_env(listen),
Children = lists:map(
fun(Address) ->
{X1,X2,X3} = os:timestamp(),
ChildName = string:join([
"Listener-",
integer_to_list(X1),
integer_to_list(X2),
integer_to_list(X3)
], ""),
?CHILD(ChildName, [Address])
end,
Addresses
),
{ok, { {one_for_one, 5, 10}, Children }}.
This allows me to dynamically generate a name for each worker and also generate the worker definitions. So:
Is this solution acceptable? It seems to me that it's not that elegant. Is there any standard solution (or best practice, etc) for this kind of use cases?
I know about the "simple_one_for_one" strategy, that allows to dynamically add workers to a supervisor. But can it also be used to dynamically generate the worker's names? Is it better (in any way) to use "simple_one_for_one" instead of my own solution that uses "one_for_one"? (again, for this particular situation).
Thanks in advance, and sorry for the long post! :)
With simple_one_for_one :
in the init function of your workers, you can register them in a table to associate a name with their PID, so you will be able to get the pid from the name.
You can use the global module (or gproc !) to associate a name with a pid. When a process dies, the name is automatically deleted by global or gproc, so when the supervisor restarts the child, the name is available.
You would pass the names in the arguments list (2nd param) of supervisor:start_child/2
Using simple_one_for_one will allow you to dynamically add more listeners after your supervisor initialization.
If you need this feature, simple_one_for_one is the good solution.
If you stick with your solution of a dynamic content inside the init function of your sup, you could clean the code like this, it may, or may not, seem more elegant :
-define(
CHILD(Name, Args),
{Name, {
?MODULE, start_listen, Args
}, permanent, 5000, worker, [?MODULE]
}
).
generate_names(Adresses) -> generate_names(Adresses, 1, []).
generate_names([], _, Acc) -> Acc;
generate_names([Addr|Addresses], ID, Acc) -> generate_names(Addresses, ID+1, [{id_name(ID), Addr}|Acc]).
id_name(ID) -> "listener-" ++ integer_to_list(ID).
init([]]) ->
{ok, Addresses} = application:get_env(listen),
Children = [?CHILD(Name, Address) || {Name, Address} <- generate_names(Addresses)],
{ok, { {one_for_one, 5, 10}, Children }}.
Or use a lists:foldl instead of all theese little functions to keep the code short.
But anyway i would pass the Adresses in the Args list of init and not call get_env inside init to keep it pure.
From your code, I know that you want to get the number of children from environment.
In the source code of worker_pool_sup.erl of rabbitmq open source project, I have read the almost similar requirement code, and the code is much elegant, I think it is helpful to you.
3 files are related, worker_pool_sup.erl, worker_pool_worker.erl, work_pool.erl.
The following code is from worker_pool_sup.erl.
init([WCount]) ->
{ok, {{one_for_one, 10, 10},
[{worker_pool, {worker_pool, start_link, []}, transient,
16#ffffffff, worker, [worker_pool]} |
[{N, {worker_pool_worker, start_link, [N]}, transient, 16#ffffffff,
worker, [worker_pool_worker]} || N <- lists:seq(1, WCount)]]}}.
For how to use it, the following code is from worker_pool.erl
get_worker_pid(WId) ->
[{WId, Pid, _Type, _Modules} | _] =
lists:dropwhile(fun ({Id, _Pid, _Type, _Modules})
when Id =:= WId -> false;
(_) -> true
end,
supervisor:which_children(worker_pool_sup)),
Pid.
I am trying to use Erlang/ets to store/update various informations by pattern matching received data. Here is the code
start() ->
S = ets:new(test,[]),
register(proc,spawn(fun() -> receive_data(S) end)).
receive_data(S) ->
receive
{see,A} -> ets:insert(S,{cycle,A}) ;
[[f,c],Fcd,Fca,_,_] -> ets:insert(S,{flag_c,Fcd,Fca});
[[b],Bd,Ba,_,_] -> ets:insert(S,{ball,Bd,Ba})
end,
receive_data(S).
Here A is cycle number, [f,c] is center flag , [b] is ball and Fcd,Fca, Bd, Ba are directions and angle of flag and ball from player.
Sender process is sending these informations. Here, pattern matching is working correctly which I checked by printing values of A, Fcd,Fca..etc. I believe there is something wrong with the use of Erlang/ets.
When I run this code I get error like this
Error in process <0.48.0> with exit value: {badarg,[{ets,insert,[16400,{cycle,7}]},{single,receive_data,1}]
Can anybody tell me what's wrong with this code and how to correct this problem?
The problem is that the owner of the ets-table is the process running the start/1 function and the default behavior for ets is to only allow the owner to write and other processes to read, aka protected. Two solutions:
Create the ets table as public
S = ets:new(test,[public]).
Set the owner to your newly created process
Pid = spawn(fun() -> receive_data(S) end,
ets:give_away(test, Pid, gift)
register(proc,Pid)
Documentation for give_away/3