Related
I need to make an address book in Erlang. I have done almost everything except for one function that's giving me problems.
My record is:
-record(contact, {fname, lname, phone=[], mail=[], city=[], street=[]}).
I have to write a function that will search through instances of contact and find all with a particular city name, and for those instances return {fname,lname} tuples. Different contacts can of course have the same cities.
When I needed similar functions for the mail and phone fields I did it this way:
findByPhone(_,[]) -> {error,"not found"};
findByPhone(Phone,[H|T]) ->
case findPhoneForUser(Phone, H#contact.phone) of
true -> {H#contact.fname, H#contact.lname};
false -> findByPhone(Phone, T)
end.
findPhoneForUser(_,[]) -> false;
findPhoneForUser(Phone, [Phone|_]) -> true;
findPhoneForUser(Phone, [_|T]) -> findPhoneForUser(Phone, T).
But both mail and phone are unique values, so whenever one is found the function is finished. For city, a search can yield multiple return values, so it must collect all matches.
How to handle this problem? I thought about list comprehensions something like:
{X,Y} || X<-H#contact.fname, Y<-H#contact.lname, City=:=H#contact.city
but it will return tuples from single ASCII codes :/
You can use a list comprehension. Assuming your address book is stored in a variable named AddressBook, and the city you're matching is stored in a variable named City, the following will work:
[{C#contact.fname, C#contact.lname} || C <- AddressBook, C#contact.city == City].
Also note that you can simplify your findByPhone function using the lists:keyfind/3 function:
findByPhone(Phone, AddressBook) ->
case lists:keyfind(Phone, #contact.phone, AddressBook) of
#contact{fname=Fname, lname=Lname} -> {Fname, Lname};
false -> {error, not_found}
end.
This works because records are tuples under the covers. The construct #contact.phone used as the second argument to lists:keyfind/3 provides the element number of the phone field of the underlying tuple. In fact, you could write a single non-exported find function supporting any unique field in your record with this approach, and then write exported functions for each such searchable field:
find(Value, Field, AddressBook) ->
case lists:keyfind(Value, Field, AddressBook) of
#contact{fname=Fname, lname=Lname} -> {Fname, Lname};
false -> {error, not_found}
end.
findByPhone(Phone, AddressBook) -> find(Phone, #contact.phone, AddressBook).
findByMail(Mail, AddressBook) -> find(Mail, #contact.mail, AddressBook).
I have this code:
-module(info).
-export([map_functions/0]).
-author("me").
map_functions() ->
{Mod,_} = code:all_loaded(),
map_functions(Mod,#{});
map_functions([H|Tail],A) ->
B = H:mod_info(exports),
map_functions(Tail,A#{H => B});
map_functions([],A) -> A.
However whenever I compile it I get a head mismatch on line 10 which is the
map_funtions([H|Tail],A) ->
I'm sure this is a very basic error but I just cannot get my head around why this does not run. It is a correct pattern match syntax [H|Tail] and the three functions with the same name but different arities are separated by commas.
Your function definition should be
map_functions() ->
{Mod,_} = code:all_loaded(),
map_functions(Mod, #{}).
map_functions([], A) -> A;
map_functions([H|Tail], A) ->
B = H:mod_info(exports),
map_functions(Tail, A#{H => B}).
The name map_functions is the same, but the arity is not. In the Erlang world that means these are two entirely different functions: map_functions/0 and map_functions/2.
Also, note that I put the "base case" first in map_functions/2 (and made the first clause's return value stick out -- breaking that to two lines is more common, but whatever). This is for three reasons: clarity, getting in the habit of writing the base case first (so you don't accidentally write infinite loops), and very often it is necessary to do this so you don't accidentally mask your base case by matching every parameter in a higher-precedence clause.
Some extended discussion on this topic is here (addressing Elixir and Erlang): Specify arity using only or except when importing function on Elixir
Function with same name but different arity are different, they are separated by dots.
code:all_loaded returns a list, so the first function should be written:
map_functions() ->
Mods = code:all_loaded(),
map_functions(Mods, #{}).
The resulting list Mods is a list of tuples of the form {ModName,BeamLocation} so the second function should be written:
map_functions([], A) -> A;
map_functions([{ModName,_}|Tail], A) ->
B = ModName:module_info(exports),
map_functions(Tail, A#{ModName => B}).
Note that you should dig into erlang libraries and try to use more idiomatic forms of code, the whole function, using list comprehension, can be written:
map_functions() ->
maps:from_list([{X,X:module_info(exports)} || {X,_} <- code:all_loaded()]).
Is there a generic way, given a complex object in Erlang, to come up with a valid function declaration for it besides eyeballing it? I'm maintaining some code previously written by someone who was a big fan of giant structures, and it's proving to be error prone doing it manually.
I don't need to iterate the whole thing, just grab the top level, per se.
For example, I'm working on this right now -
[[["SIP",47,"2",46,"0"],32,"407",32,"Proxy Authentication Required","\r\n"],
[{'Via',
[{'via-parm',
{'sent-protocol',"SIP","2.0","UDP"},
{'sent-by',"172.20.10.5","5060"},
[{'via-branch',"z9hG4bKb561e4f03a40c4439ba375b2ac3c9f91.0"}]}]},
{'Via',
[{'via-parm',
{'sent-protocol',"SIP","2.0","UDP"},
{'sent-by',"172.20.10.15","5060"},
[{'via-branch',"12dee0b2f48309f40b7857b9c73be9ac"}]}]},
{'From',
{'from-spec',
{'name-addr',
[[]],
{'SIP-URI',
[{userinfo,{user,"003018CFE4EF"},[]}],
{hostport,"172.20.10.11",[]},
{'uri-parameters',[]},
[]}},
[{tag,"b7226ffa86c46af7bf6e32969ad16940"}]}},
{'To',
{'name-addr',
[[]],
{'SIP-URI',
[{userinfo,{user,"3966"},[]}],
{hostport,"172.20.10.11",[]},
{'uri-parameters',[]},
[]}},
[{tag,"a830c764"}]},
{'Call-ID',"90df0e4968c9a4545a009b1adf268605#172.20.10.15"},
{'CSeq',1358286,"SUBSCRIBE"},
["date",'HCOLON',
["Mon",44,32,["13",32,"Jun",32,"2011"],32,["17",58,"03",58,"55"],32,"GMT"]],
{'Contact',
[[{'name-addr',
[[]],
{'SIP-URI',
[{userinfo,{user,"3ComCallProcessor"},[]}],
{hostport,"172.20.10.11",[]},
{'uri-parameters',[]},
[]}},
[]],
[]]},
["expires",'HCOLON',3600],
["user-agent",'HCOLON',
["3Com",[]],
[['LWS',["VCX",[]]],
['LWS',["7210",[]]],
['LWS',["IP",[]]],
['LWS',["CallProcessor",[['SLASH',"v10.0.8"]]]]]],
["proxy-authenticate",'HCOLON',
["Digest",'LWS',
["realm",'EQUAL',['SWS',34,"3Com",34]],
[['COMMA',["domain",'EQUAL',['SWS',34,"3Com",34]]],
['COMMA',
["nonce",'EQUAL',
['SWS',34,"btbvbsbzbBbAbwbybvbxbCbtbzbubqbubsbqbtbsbqbtbxbCbxbsbybs",
34]]],
['COMMA',["stale",'EQUAL',"FALSE"]],
['COMMA',["algorithm",'EQUAL',"MD5"]]]]],
{'Content-Length',0}],
"\r\n",
["\n"]]
Maybe https://github.com/etrepum/kvc
I noticed your clarifying comment. I'd prefer to add a comment myself, but don't have enough karma. Anyway, the trick I use for that is to experiment in the shell. I'll iterate a pattern against a sample data structure until I've found the simplest form. You can use the _ match-all variable. I use an erlang shell inside an emacs shell window.
First, bind a sample to a variable:
A = [{a,b},[{c,d}, {e,f}]].
Now set the original structure against the variable:
[{a,b},[{c,d},{e,f}]] = A.
If you hit enter, you'll see they match. Hit alt-p (forget what emacs calls alt, but it's alt on my keyboard) to bring back the previous line. Replace some tuple or list item with an underscore:
[_,[{c,d},{e,f}]].
Hit enter to make sure you did it right and they still match. This example is trivial, but for deeply nested, multiline structures it's trickier, so it's handy to be able to just quickly match to test. Sometimes you'll want to try to guess at whole huge swaths, like using an underscore to match a tuple list inside a tuple that's the third element of a list. If you place it right, you can match the whole thing at once, but it's easy to misread it.
Anyway, repeat to explore the essential shape of the structure and place real variables where you want to pull out values:
[_, [_, _]] = A.
[_, _] = A.
[_, MyTupleList] = A. %% let's grab this tuple list
[{MyAtom,b}, [{c,d}, MyTuple]] = A. %% or maybe we want this atom and tuple
That's how I efficiently dissect and pattern match complex data structures.
However, I don't know what you're doing. I'd be inclined to have a wrapper function that uses KVC to pull out exactly what you need and then distributes to helper functions from there for each type of structure.
If I understand you correctly you want to pattern match some large datastructures of unknown formatting.
Example:
Input: {a, b} {a,b,c,d} {a,[],{},{b,c}}
function({A, B}) -> do_something;
function({A, B, C, D}) when is_atom(B) -> do_something_else;
function({A, B, C, D}) when is_list(B) -> more_doing.
The generic answer is of course that it is undecidable from just data to know how to categorize that data.
First you should probably be aware of iolists. They are created by functions such as io_lib:format/2 and in many other places in the code.
One example is that
[["SIP",47,"2",46,"0"],32,"407",32,"Proxy Authentication Required","\r\n"]
will print as
SIP/2.0 407 Proxy Authentication Required
So, I'd start with flattening all those lists, using a function such as
flatten_io(List) when is_list(List) ->
Flat = lists:map(fun flatten_io/1, List),
maybe_flatten(Flat);
flatten_io(Tuple) when is_tuple(Tuple) ->
list_to_tuple([flatten_io(Element) || Element <- tuple_to_list(Tuple)];
flatten_io(Other) -> Other.
maybe_flatten(L) when is_list(L) ->
case lists:all(fun(Ch) when Ch > 0 andalso Ch < 256 -> true;
(List) when is_list(List) ->
lists:all(fun(X) -> X > 0 andalso X < 256 end, List);
(_) -> false
end, L) of
true -> lists:flatten(L);
false -> L
end.
(Caveat: completely untested and quite inefficient. Will also crash for inproper lists, but you shouldn't have those in your data structures anyway.)
On second thought, I can't help you. Any data structure that uses the atom 'COMMA' for a comma in a string should be taken out and shot.
You should be able to flatten those things as well and start to get a view of what you are looking at.
I know that this is not a complete answer. Hope it helps.
Its hard to recommend something for handling this.
Transforming all the structures in a more sane and also more minimal format looks like its worth it. This depends mainly on the similarities in these structures.
Rather than having a special function for each of the 100 there must be some automatic reformatting that can be done, maybe even put the parts in records.
Once you have records its much easier to write functions for it since you don't need to know the actual number of elements in the record. More important: your code won't break when the number of elements changes.
To summarize: make a barrier between your code and the insanity of these structures by somehow sanitizing them by the most generic code possible. It will be probably a mix of generic reformatting with structure speicific stuff.
As an example already visible in this struct: the 'name-addr' tuples look like they have a uniform structure. So you can recurse over your structures (over all elements of tuples and lists) and match for "things" that have a common structure like 'name-addr' and replace these with nice records.
In order to help you eyeballing you can write yourself helper functions along this example:
eyeball(List) when is_list(List) ->
io:format("List with length ~b\n", [length(List)]);
eyeball(Tuple) when is_tuple(Tuple) ->
io:format("Tuple with ~b elements\n", [tuple_size(Tuple)]).
So you would get output like this:
2> eyeball({a,b,c}).
Tuple with 3 elements
ok
3> eyeball([a,b,c]).
List with length 3
ok
expansion of this in a useful tool for your use is left as an exercise. You could handle multiple levels by recursing over the elements and indenting the output.
Use pattern matching and functions that work on lists to extract only what you need.
Look at http://www.erlang.org/doc/man/lists.html:
keyfind, keyreplace, L = [H|T], ...
Lets say I have a record:
-record(foo, {bar}).
What I would like to do is to be able to pass the record name to a function as a parameter, and get back a new record. The function should be generic so that it should be able to accept any record, something like this.
make_record(foo, [bar], ["xyz"])
When implementing such a function I've tried this:
make_record(RecordName, Fields, Values) ->
NewRecord = #RecordName{} %% this line gives me an error: syntax error before RecordName
Is it possible to use the record name as a parameter?
You can't use the record syntax if you don't have access to the record during compile time.
But because records are simply transformed into tuples during compilation it is really easy to construct them manually:
-record(some_rec, {a, b}).
make_record(Rec, Values) ->
list_to_tuple([Rec | Values]).
test() ->
R = make_record(some_rec, ["Hej", 5]), % Dynamically create record
#some_rec{a = A, b = B} = R, % Access it using record syntax
io:format("a = ~p, b = ~p~n", [A, B]).
Alternative solution
Or, if you at compile time make a list of all records that the function should be able to construct, you can use the field names also:
%% List of record info created with record_info macro during compile time
-define(recs,
[
{some_rec, record_info(fields, some_rec)},
{some_other_rec, record_info(fields, some_other_rec)}
]).
make_record_2(Rec, Fields, Values) ->
ValueDict = lists:zip(Fields, Values),
% Look up the record name and fields in record list
Body = lists:map(
fun(Field) -> proplists:get_value(Field, ValueDict, undefined) end,
proplists:get_value(Rec, ?recs)),
list_to_tuple([Rec | Body]).
test_2() ->
R = make_record_2(some_rec, [b, a], ["B value", "A value"]),
#some_rec{a = A, b = B} = R,
io:format("a = ~p, b = ~p~n", [A, B]).
With the second version you can also do some verification to make sure you are using the right fields etc.
Other tips
Other useful constructs to keep in mind when working with records dynamically is the #some_rec.a expression which evaluates to the index of the a field in some_recs, and the element(N, Tuple) function which given a tuple and an index returns the element in that index.
This is not possible, as records are compile-time only structures. At compilation they are converted into tuples. Thus the compiler needs to know the name of the record, so you cannot use a variable.
You could also use some parse-transform magic (see exprecs) to create record constructors and accessors, but this design seems to go in the wrong direction.
If you need to dynamically create record-like things, you can use some structures instead, like key-value lists, or dicts.
To cover all cases: If you have fields and values but don't necessarily have them in the correct order, you could make your function take in the result of record_info(fields, Record), with Record being the atom of the record you want to make. Then it'll have the ordered field names to work with. And a record is just a tuple with its atom name in the first slot, so you can build it that way. Here's how I build an arbitrary shallow record from a JSON string (not thoroughly tested and not optimized, but tested and working):
% Converts the given JSON string to a record
% WARNING: Only for shallow records. Won't work for nested ones!
%
% Record: The atom representing the type of record to be converted to
% RecordInfo: The result of calling record_info(fields, Record)
% JSON: The JSON string
jsonToRecord(Record, RecordInfo, JSON) ->
JiffyList = element(1, jiffy:decode(JSON)),
Struct = erlang:make_tuple(length(RecordInfo)+1, ""),
Struct2 = erlang:setelement(1, Struct, Record),
recordFromJsonList(RecordInfo, Struct2, JiffyList).
% private methods
recordFromJsonList(_RecordInfo, Struct, []) -> Struct;
recordFromJsonList(RecordInfo, Struct, [{Name, Val} | Rest]) ->
FieldNames = atomNames(RecordInfo),
Index = index_of(erlang:binary_to_list(Name), FieldNames),
recordFromJsonList(RecordInfo, erlang:setelement(Index+1, Struct, Val), Rest).
% Converts a list of atoms to a list of strings
%
% Atoms: The list of atoms
atomNames(Atoms) ->
F = fun(Field) ->
lists:flatten(io_lib:format("~p", [Field]))
end,
lists:map(F, Atoms).
% Gets the index of an item in a list (one-indexed)
%
% Item: The item to search for
% List: The list
index_of(Item, List) -> index_of(Item, List, 1).
% private helper
index_of(_, [], _) -> not_found;
index_of(Item, [Item|_], Index) -> Index;
index_of(Item, [_|Tl], Index) -> index_of(Item, Tl, Index+1).
Brief explanation: The JSON represents some key:value pairs corresponding to field:value pairs in the record we're trying to build. We might not get the key:value pairs in the correct order, so we need the list of record fields passed in so we can insert the values into their correct positions in the tuple.
I'm developing an Erlang system and having reoccurring problems with the fact that records are compile-time pre-processor macros (almost), and that they cant be manipulated at runtime...
basically, I'm working with a property pattern, where properties are added at run-time to objects on the front-end (AS3). Ideally, I would reflect this with a list on the Erlang side, since its a fundamental data type, but then using records in QCL [to query ETS tables] would not be possible since to use them I have to specifically say which record property I want to query over... I have at least 15 columns in the larges table, so listing them all in one huge switch statement (case X of) is just plain ugly.
does anyone have any ideas how to elegantly solve this? maybe some built-in functions for creating tuples with appropriate signatures for use in pattern matching (for QLC)?
thanks
It sounds like you want to be able to do something like get_record_field(Field, SomeRecord) where Field is determined at runtime by user interface code say.
You're right in that you can't do this in standard erlang as records and the record_info function are expanded and eliminated at compile time.
There are a couple of solutions that I've used or looked at. My solution is as follows: (the example gives runtime access to the #dns_rec and #dns_rr records from inet_dns.hrl)
%% Retrieves the value stored in the record Rec in field Field.
info(Field, Rec) ->
Fields = fields(Rec),
info(Field, Fields, tl(tuple_to_list(Rec))).
info(_Field, _Fields, []) -> erlang:error(bad_record);
info(_Field, [], _Rec) -> erlang:error(bad_field);
info(Field, [Field | _], [Val | _]) -> Val;
info(Field, [_Other | Fields], [_Val | Values]) -> info(Field, Fields, Values).
%% The fields function provides the list of field positions
%% for all the kinds of record you want to be able to query
%% at runtime. You'll need to modify this to use your own records.
fields(#dns_rec{}) -> fields(dns_rec);
fields(dns_rec) -> record_info(fields, dns_rec);
fields(#dns_rr{}) -> fields(dns_rr);
fields(dns_rr) -> record_info(fields, dns_rr).
%% Turns a record into a proplist suitable for use with the proplists module.
to_proplist(R) ->
Keys = fields(R),
Values = tl(tuple_to_list(R)),
lists:zip(Keys,Values).
A version of this that compiles is available here: rec_test.erl
You can also extend this dynamic field lookup to dynamic generation of matchspecs for use with ets:select/2 or mnesia:select/2 as shown below:
%% Generates a matchspec that does something like this
%% QLC psuedocode: [ V || #RecordKind{MatchField=V} <- mnesia:table(RecordKind) ]
match(MatchField, RecordKind) ->
MatchTuple = match_tuple(MatchField, RecordKind),
{MatchTuple, [], ['$1']}.
%% Generates a matchspec that does something like this
%% QLC psuedocode: [ T || T <- mnesia:table(RecordKind),
%% T#RecordKind.Field =:= MatchValue]
match(MatchField, MatchValue, RecordKind) ->
MatchTuple = match_tuple(MatchField, RecordKind),
{MatchTuple, [{'=:=', '$1', MatchValue}], ['$$']}.
%% Generates a matchspec that does something like this
%% QLC psuedocode: [ T#RecordKind.ReturnField
%% || T <- mnesia:table(RecordKind),
%% T#RecordKind.MatchField =:= MatchValue]
match(MatchField, MatchValue, RecordKind, ReturnField)
when MatchField =/= ReturnField ->
MatchTuple = list_to_tuple([RecordKind
| [if F =:= MatchField -> '$1'; F =:= ReturnField -> '$2'; true -> '_' end
|| F <- fields(RecordKind)]]),
{MatchTuple, [{'=:=', '$1', MatchValue}], ['$2']}.
match_tuple(MatchField, RecordKind) ->
list_to_tuple([RecordKind
| [if F =:= MatchField -> '$1'; true -> '_' end
|| F <- fields(RecordKind)]]).
Ulf Wiger has also written a parse_transform, Exprecs, that more or less does this for you automagically. I've never tried it, but Ulf's code is usually very good.
I solve this problem (in development) by use the parse transform tools to read the .hrl files and generate helper functions.
I wrote a tutorial on it at Trap Exit.
We use it all the time to generate match specs. The beauty is that you don't need to know anything about the current state of the record at development time.
However once you are in production things change! If your record is the basis of a table (as opposed to the definition of a field in a table) then changing an underlying record is more difficult (to put it mildly!).
I'm not sure I fully understand your Problem but I have moved from records to proplists in most cases. They are much more flexible and much slower. Using (d)ets I usually use a few record fields for coarse selection and then check the proplists on the remaining records for detailed selection.