I am trying to insert something into cassandra 0.7.6 from Erlang R14B02 (through thrift 0.6.1)
I am doing the following:
Read record definitions
rr(cassandra_types).
Connect to cassandra
{ok, C}=thrift_client_util:new("localhost", 9160, cassandra_thrift,[{strict_read, false}, {strict_write, false}, {framed, true}]).
Try to insert a value (timestamp=1, 2=Quorum)
Reply1 = thrift_client:call(C, 'insert', ["existing_keyspace", "new_key",#columnPath{column_family = "existing_column_family", column = "existing_column"}, "new_value",1,2]).
But nr3 gives me a bad_args error (1 and 2 work perfectly). What would be the right arguments?
What API information there is for unsupported languages is largely in their Cassandra Thrift API documentation.
In Cassandra 0.7, you don't supply the keyspace for most operations, so insert just takes [Key, ColumnPath, Column, ConsistencyLevel]. You need to call set_keyspace before attempting the insert. The insert in erlang would be
Reply1 = thrift_client:call(C, 'insert',
[SomeKey,
#columnPath{column_family = "existing_column_family",
column = "existing_column"},
#column{name="existing_column",
value="new_value",timestamp=1},
?cassandra_ConsistencyLevel_QUORUM]).
Your example is missing the row key for the insert I think too.
As an aside, make sure you always update the value of C - it changes after every thrift_client call.
?cassandra_ConsistencyLevel_QUORUM is 2 from memory.
This might help, although there is no erlang-specific code: http://wiki.apache.org/cassandra/ThriftExamples
Related
I'm new to erlang, and am running into an error with records in one of my modules. I'm emulating ships inside of a shipping_state, and I want to create a simple function that will print the ship id, name, and container cap of a certain ship, based on it's ID. I utilized list:keyfind, as I believe it will help, but perhaps I am not using it correctly. I have a .hrl file that contains the record declarations, and a .erl file with the function and initialization of my #shipping_state.
shipping.erl:
-module(shipping).
-compile(export_all).
-include_lib("./shipping.hrl").
get_ship(Shipping_State, Ship_ID) ->
{id, name, containercap} = list:keyfind(Ship_ID, 1, Shipping_State#shipping_state.ships).
shipping.hrl:
-record(ship, {id, name, container_cap}).
-record(container, {id, weight}).
-record(shipping_state,
{
ships = [],
containers = [],
ports = [],
ship_locations = [],
ship_inventory = maps:new(),
port_inventory = maps:new()
}
).
-record(port, {id, name, docks = [], container_cap}).
Result:
shipping:get_ship(shipping:init(),1).
** exception error: {badrecord,shipping_state}
in function shipping:get_ship/2 (shipping.erl, line 18)
I'd like to say that keyfind should work, and perhaps when I create the tuple {id, name, containercap}, something is wrong with the syntax there, but if I need to completely rethink how I would go about doing this problem, any assistance would be greatly appreciated.
Edit,
I've modified my code to follow Alexey's suggestions, however, there still appears to be the same error. Any further insights?
get_ship(Shipping_State, Ship_ID) ->
{ship, Id, Name, Containercap} = list:keyfind(Ship_ID, 2,
Shipping_State#shipping_state.ships),
io:format("id = ~w, name = ~s, container cap = ~w",[Id, Name, Containercap]).
See Internal Representation of Records: #ship{id=1,name="Santa Maria",container_cap=20} becomes {ship, 1, "Santa Maria", 20}, so the id is the 2nd element, not the first one.
{id, name, containercap} = ...
should be
#ship{id=Id, ...} = ...
or
{ship, Id, Name, Containercap} = ...
Your current code would only succeed if keyfind returned a tuple of 3 atoms.
The error {badrecord,shipping_state} is telling you that the code of get_ship expects its first argument to be a #shipping_state, but you pass {ok, #shipping_state{...}} (the result of init).
Records were added to the Erlang language because dealing with tuples fields by number was error-prone, especially as code changed during development and tuple fields were added, changed, or dropped. Don't use numbers to identify record fields, and don't treat records using their underlying tuple representation, as both work against the purpose of records and are unnecessary.
In your code, rather than using record field numbers with lists:keyfind/3, use the record names themselves. I've revised your get_ship/2 function to do this:
get_ship(Shipping_State, Ship_ID) ->
#ship{id=ID, name=Name, container_cap=ContainerCap} = lists:keyfind(Ship_ID, #ship.id, Shipping_State#shipping_state.ships),
io:format("id = ~w, name = ~s, container cap = ~w~n",[ID, Name, ContainerCap]).
The syntax #<record_name>.<record_field_name> provides the underlying record field number. In the lists:keyfind/3 call above, #ship.id provides the field number for the id field of the ship record. This will continue to work correctly even if you add fields to the record, and unlike a raw number it will cause a compilation error should you decide to drop that field from the record at some point.
If you load your record definitions into your shell using the rr command, you can see that #ship.id returns the expected field number:
1> rr("shipping.hrl").
[container,port,ship,shipping_state]
2> #ship.id.
2
With the additional repairs to your function above to handle the returned record correctly, it now works as expected, as this shell session shows:
3> {ok, ShippingState} = shipping:init().
{ok,{shipping_state,[{ship,1,"Santa Maria",20},
{ship,2,"Nina",20},
{ship,3,"Pinta",20},
{ship,4,"SS Minnow",20},
{ship,5,"Sir Leaks-A-Lot",20}],
[{container,1,200},
...
4> shipping:get_ship(ShippingState, 1).
id = 1, name = Santa Maria, container cap = 20
ok
Alexey's answer answers your question, in particular the 3rd point. I just want to suggest an improvement to your keyfind call. You need to pass the tuple index to it, but you can use record syntax to get that index without hard-coding it, like this:
list:keyfind(Ship_ID, #ship.id, Shipping_State#shipping_state.ships),
#ship.id returns the index of the id field, in this case 2. This makes it easier to read the code - no need to wonder what the constant 2 is for. Also, if for whatever reason you change the order of fields in the ship record, this code will still compile and do the right thing.
We recently upgraded to Rails 4.2 from Rails 4.1 and are seeing problems with using Arel + Activerecord because we're getting this type of error:
ActiveRecord::StatementInvalid: PG::ProtocolViolation: ERROR: bind message supplies 0 parameters, but prepared statement "" requires 8
Here's the code that is breaking:
customers = Customer.arel_table
ne_subquery = ImportLog.where(
importable_type: Customer.to_s,
importable_id: customers['id'],
remote_type: remote_type.to_s.singularize,
destination: 'hello'
).exists.not
first = Customer.where(ne_subquery).where(company_id: #company.id)
second = Customer.joins(:import_logs).merge(
ImportLog.where(
importable_type: Customer.to_s,
importable_id: customers['id'],
remote_type: remote_type.to_s.singularize,
status: 'pending',
destination: 'hello',
remote_id: nil
)
).where(company_id: #company.id)
Customer.from(
customers.create_table_alias(
first.union(second),
Customer.table_name
)
)
We figured out how to solve the first part of the query (running into the same rails bug of not having bindings) by moving the exists.not to be within Customer.where like so:
ne_subquery = ImportLog.where(
importable_type: Customer.to_s,
importable_id: customers['id'],
destination: 'hello'
)
first = Customer.where("NOT (EXISTS (#{ne_subquery.to_sql}))").where(company_id: #company.id)
This seemed to work but we ran into the same issue with this line of code:
first.union(second)
whenever we run this part of the query, the bindings get lost. first and second are both active record objects but as soon as we "union" them, they lose the bindings are become arel objects.
We tried cycling through the query and manually replacing the bindings but couldn't seem to get it working properly. What should we do instead?
EDIT:
We also tried extracting the bind values from first and second, and then manually replacing them in the arel object like so:
union.grep(Arel::Nodes::BindParam).each_with_index do |bp, i|
bv = bind_values[i]
bp.replace(Customer.connection.substitute_at(bv, i))
end
However, it fails because:
NoMethodError: undefined method `replace' for #<Arel::Nodes::BindParam:0x007f8aba6cc248>
This was a solution suggested in the rails github repo.
I know this question is a bit old, but the error sounded familiar. I had some notes and our solution in a repository, so I thought I'd share.
The error we were receiving was:
PG::ProtocolViolation: ERROR: bind message supplies 0 parameters, but
prepared statement "" requires 1
So as you can see, our situation is a bit different. We didn't have 8 bind values. However, our single bind value was still being clobbered. I changed the naming of things to keep it general.
first_level = Blog.all_comments
second_level = Comment.where(comment_id: first_level.select(:id))
third_level = Comment.where(comment_id: second_level.select(:id))
Blog.all_comments is where we have the single bind value. That's the piece we're losing.
union = first_level.union second_level
union2 = Comment.from(
Comment.arel_table.create_table_alias union, :comments
).union third_level
relation = Comment.from(Comment.arel_table.create_table_alias union2, :comments)
We created a union much like you except that we needed to union three different queries.
To get the lost bind values at this point, we did a simple assignment. In the end, this is a little simpler of a case than yours. However, it may be helpful.
relation.bind_values = first_level.bind_values
relation
By the way, here's the GitHub issue we found while working on this. It doesn't appear to have any updates since this question was posted though.
Iam new in erlang and trying to get data from mysql server:
<erl>
out(A) ->
application:start(odbc),
ConnString =
"Driver={MySQL ODBC 5.2 ANSI Driver};" ++
"Server=127.0.0.1;Database=teamsDatabase;" ++
"User=root;Password=1q2w3e;" ++
"Option=3;",
{ok, Conn} = odbc:connect(ConnString, []),
Results = odbc:sql_query(Conn, "select team_name from teams limit 2"),
{ehtml,
[{h4,[], "The database result:"},
{hr},
{html, lists:map(fun(X) -> {Tname} = X, io_lib:format("ID: ~p ", [Tname]) end, Results)}]}.
of course, I get the error:
ERROR erlang code threw an uncaught exception:
File: c:/yaws/zero.yaws:39
Class: error
Exception: function_clause
Req: {http_request,'GET',{abs_path,"/zero.yaws"},{1,1}}
Stack: [{lists,map,
[#Fun,
{selected,["team_name"],[{"Team 1"},{"Team 2"}]}],
[{file,"lists.erl"},{line,1223}]},
How i can output my data? It`s looks like this:
{selected,["team_name"],[{"Team 1"},{"Team 2"}]}
This is list? or..?
I found this:
Output data of Erlang List as a HTML in Yaws
But it is did not work for me.
The error you're getting is due to passing a tuple rather than a list as the second argument to lists:map/2.
If you're using ehtml to return results, you could do this to just print the whole result as a string:
{ehtml,
[{h4, [], "The database result:"},
{hr},
{p, [], io_lib:format("~p", [Results])}]}
But of course that's less than ideal as it exposes Erlang-formatted terms in your web page. A better approach might be:
{selected, [Selector], Results} = odbc:sql_query(Conn, "select team_name from teams limit 2"),
{ehtml,
[{h4, [], "The database result:"},
{hr},
[{p, [], [Selector, ": ", Val]} || Val <- Results]]}
which results in the following HTML:
<h4>The database result:</h4><hr />
<p>team_name: Team 1</p>
<p>team_name: Team 2</p>
Notice that this second approach pattern-matches the results coming back from the database query to help prepare the results for use in forming the ehtml. You can change the details of this approach to format the results as you wish.
BTW you should use another approach to establishing your database connection, since putting it into the out/1 function of a .yaws page means it's going to be run every time a client requests that page. For something simple you might create a small Erlang application with one supervisor that watches a gen_server that connects to the database, where your gen_server callback module has an API function to return the connection for use in your out/1 function. This application can be started when Yaws starts via the runmod feature described in the Yaws documentation. The nice thing about this is that it can cleanly connect and disconnect from the database during application start and stop respectively, but note that it gives you only a single connection. For a more scalable approach, you can probably find database connection pooling modules on github or other sites that you can use instead.
I'm currently trying to get my head wrap around Cassandra/thrift with Erlang...
I have a Column Family named "mq" (as in message queue)...
I would like to have a row per user (with an user_id), each message would be a new column with timestamp for name and the message as the value.
Here's in Cassandra-cli what I'm doing :
create keyspace my_keyspace;
use my_keyspace;
create column family mq with comparator=UTF8Type and key_validation_class=UTF8Type;
%% Adding to user_id (00000001) the message "Hello World!"
set mq['00000001']['1336499385041308'] = 'Hello Wold';
Everything works great with Cassandra-cli
However, When I'm trying to insert from Erlang, I'm running into some issue :
1>rr(cassandra_types).
2>{ok, C}=thrift_client_util:new("localhost", 9160, cassandra_thrift,[{framed, true}]).
3>thrift_client:call(C, 'set_keyspace', ["peeem"]).
4>thrift_client:call(C,'insert',["00000001",
#columnPath{column_family="mq"},
#column{name="1336499385041308", value="Hello World!"},
1
]
).
Here's the error :
{error,{bad_args,insert,
["00000001",
#columnPath{column_family = "mq",super_column = undefined,
column = undefined},
#column{name = "1336499385041308",value = "Hello World!",
timestamp = undefined,ttl = undefined},1]}}}
Any help would be appreciated...
EDIT 1 :
I have found out that it should be (as it works for someone else) :
thrift_client:call(C,'insert', ["00000001", #columnParent{column_family="mq"}, #column{name="123456",value="Hello World!"}, 2]).
Here's the related error message :
** exception error: no match of right hand side value {{protocol,thrift_binary_protocol,
{binary_protocol,{transport,thrift_framed_transport,
{framed_transport,{transport,thrift_socket_transport,
{data,#Port<0.730>,infinity}},
[],[]}},
true,true}},
{error,closed}}
in function thrift_client:send_function_call/3 (src/thrift_client.erl, line 83)
in call from thrift_client:call/3 (src/thrift_client.erl, line 40)
Ok I have found out what was the issue, or issues to be correct...
Here's my code in order to connect and insert...
rr(cassandra_types).
{ok, C}=thrift_client_util:new("localhost", 9160, cassandra_thrift,[{framed, true}]).
{C1, _} = thrift_client:call(C, 'set_keyspace', ["my_keyspace"]).
thrift_client:call(C1,'insert', ["00000001", #columnParent{column_family="mq"}, #column{name="1234567",value="Hello World !", timestamp=0}, 2]).
In fact We should insert into the new thrift_client that 'set_keyspace' returns... apparently for every call done through thrift a new thrift_client is generated and should be used.
Further more, We should use columnParent instead of columnPath (not sure why yet, it just works). And timestamp in the #column is mandatory...
I hope this helps someone else.
I recommend checking out https://github.com/ostinelli/erlcassa instead of reinventing the Thrift-wrestling wheel.
I am having issues with multiget_slice cassandra thrift interface in erlang. I am able to get data back from multiget_count for the same set of keys however multiget_slice returns an empty list. I am using thrift 0.8.0 and cassandra 1.0.6. Here is my code
Keys = [<<"key1">>, <<"key2">>],
ColumnParent = #columnParent{column_family=ColumnFamily},
SliceRange = #sliceRange{start="", finish="", reversed=false, count=2147483647},
SlicePredicate = #slicePredicate{slice_range=SliceRange, column_names=undefined},
{ok, Conn} = thrift_client_util:new(Host, Port, cassandra_thrift, [{framed, true}]), ok,
{Conn2, {ok, ok}} = thrift_client:call(Conn, set_keyspace, [Keyspace]),
{NewCon, Response} = thrift_client:call(Conn2, multiget_slice, [Keys, ColumnParent, SlicePredicate, 1]),
Response returns the following: {ok, []}
However running multiget_count on the same set of keys yield an appropriate result. What am I doing wrong here?
Seems like you have to specify some non-empty start and finish in SliceRange.