How to query over range of range key of dynamodb? - ruby-on-rails

I'm using 'aws-sdk', '~> 2.6.44'. I've an activities table where I store all the activities performed by a user.
params = {
table_name: 'activities', # required
key_schema: [ # required
{
attribute_name: 'actor', # required User.1
key_type: 'HASH', # required, accepts HASH, RANGE
},
{
attribute_name: 'created_at', # timestamp
key_type: 'RANGE'
}
],....
I want to query this table with all the activities performed by user in past 1 day. Looks like the AWS documentation site has the documentation for SDK version 3.
tableName = 'activities'
params = {
table_name: tableName,
key_condition_expression: "#user = :actor and #time between :start_time and :end_time",
expression_attribute_names: {
"#user" => 'actor',
"#time" => "created_at"
},
expression_attribute_values: {
actor: 'User.1',
":start_time" => (Time.now - 1.days).to_i,
":end_time" => (Time.now + 1.days).to_i
}
}
DynamodbClient.client.get_item(params)
# Throws: ArgumentError: no such member :key_condition_expression
I tried with filter expression:
tableName = 'activities'
params = {
table_name: tableName,
key: {
actor: 'User.1'
},
filter_expression: "created_at BETWEEN (:id1, :id2)",
expression_attribute_values: { ":id1" => (Time.now - 1.days).to_i,":id2" => (Time.now + 1.days).to_i},
projection_expression: "actor"
}
DynamodbClient.client.get_item(params)
# Throws ArgumentError: no such member :filter_expression
What should be right way to query DynamoDB table with a ranged option for range key?

Looks like I should use query if I'm not trying to retrieve a specific record.
Following query worked:
tableName = 'activities'
params = {
table_name: tableName,
key_condition_expression: "#user = :actor and #time between :start_time and :end_time",
expression_attribute_names: {
"#user" => 'actor',
"#time" => "created_at"
},
expression_attribute_values: {
":actor" => 'User.1',
":start_time" => (Time.now - 1.days).to_i,
":end_time" => (Time.now + 1.days).to_i
}
}
DynamodbClient.client.query(params)
=> #<struct Aws::DynamoDB::Types::QueryOutput items=[{"actor"=>"User.1", "action"=>"Like", "created_at"=>#<BigDecimal:7fa6418b86e0,'0.150683976E10',18(27)>, "source"=>"FeedSource.661", "body"=>{"id"=>#<BigDecimal:7fa6418b82f8,'0.72E2',9(18)>}, "target"=>"FeedEntry.8419"}], count=1, scanned_count=1, last_evaluated_key=nil, consumed_capacity=nil>
:)

Related

Grabbing a specific hash in an array that meets a specific criteria

I have a huge array full of a bunch of hashes. What I need to do is single out one index hash from the array that meets a specific criteria. (doing this due to an rspec test, but having trouble singling out one of them)
My array is like this
[
{
"name" => "jon doe",
"team" => "team2",
"price" => 2000,
"eligibility_settings" => {}
},
{
"name" => "jonny doe",
"team" => "team1",
"value" => 2000,
"eligibility_settings" => {
"player_gender" => male,
"player_max_age" => 26,
"player_min_age" => 23,
"established_union_only" => true
}
},
{
"name" => "jonni doe",
"team" => "team3",
"price" => 2000,
"eligibility_settings" => {}
},
]
I need to single out the second one, based on its eligibility settings. I just took three of them from my array, have lots more, so simple active record methods like (hash.second) won't work in this instance.
I've tried things like
players.team.map(&:hash).find{ |x| x[ 'eligibility_settings?' ] == true}
However when I try this, I get a nil response. (which is odd)
I've also looked into using the ruby detect method, which hasn't gotten me anywhere either
Players.team.map(&:hash).['hash.seligibiltiy_settings'].detect { true }
Would anybody have any idea what to do with this one?
Notes
players.team.map(&:hash).find{ |x| x[ 'eligibility_settings?' ] == true}
Players.team.map(&:hash).['hash.seligibiltiy_settings'].detect { true }
Is is players or Players ?
Why is it plural?
If you can call map on team, it probably should be plural
Why do you convert to a hash?
eligibility_settings? isn't a key in your hash. eligibility_settings is
eligibility_settings can be a hash, but it cannot be true
If you want to check if it isn't empty, use !h['eligibility_settings'].empty?
Possible solution
You could use :
data = [
{
'name' => 'jon doe',
'team' => 'team2',
'price' => 2000,
'eligibility_settings' => {}
},
{
'name' => 'jonny doe',
'team' => 'team1',
'value' => 2000,
'eligibility_settings' => {
'player_gender' => 'male',
'player_max_age' => 26,
'player_min_age' => 23,
'established_union_only' => true
}
},
{
'name' => 'jonni doe',
'team' => 'team3',
'price' => 2000,
'eligibility_settings' => {}
}
]
p data.find { |h| !h['eligibility_settings'].empty? }
# {"name"=>"jonny doe", "team"=>"team1", "value"=>2000, "eligibility_settings"=>{"player_gender"=>"male", "player_max_age"=>26, "player_min_age"=>23, "established_union_only"=>true}}
If h['eligibility_settings'] can be nil, you can use :
data.find { |h| !h['eligibility_settings'].blank? }
or
data.find { |h| h['eligibility_settings'].present? }

Finding all instances that match value within json stored in Table in Rails

I have a table "Transfer" in my database. In this table transfer I have a column "archive" in which I store a Json object.
So I have something like that:
archive:{
"AuthorId"=>"6621381"
}
My goal is to find all the transfers where "AuthorId"=>"6621381". Is it possible to do that with rails ?
Something that looks like:
Transfer.where(archive: {"AuthorId" => "6621381"})
Use the ->> operator to access the object field as text :
Transfer.where("archive ->> 'AuthorId' = ?", "123")
=> [0] #<Transfer:0x00000002903a60> {
:archive => {
"AuthorId" => "123"
}
}
]
It also works with other operators, such as LIKE/ILIKE :
Transfer.where("archive ->> 'AuthorId' ILIKE ?", "12%")
=> [
[0] #<Transfer:0x00000002893058> {
:archive => {
"AuthorId" => "123"
}
},
[1] #<Transfer:0x00000002892c98> {
:archive => {
"AuthorId" => "124"
}
}
]

How to use filters in Magento SOAP API v1 with Rails 4 and Savon?

I'm building an app in Rails 4 using the Magento SOAP API v1 and Savon gem. Right now I am trying to get all orders with a status of pending. To hook into the API I am using this code:
class MagentoAPI
def self.call method, options={}
response = ##soap_client.request :call do
if options.empty?
soap.body = { :session => ##soap_session, :method => method }
elsif options[:string]
soap.body = { :session => ##soap_session, :method => method, :arguments => [options[:string]] }
else
puts options
soap.body = { :session => ##soap_session, :method => method, :arguments => options }
end
end
if response.success?
# listing found products
final = []
call_return = response[:call_response][:call_return]
return [] if call_return[:item].nil?
raw = call_return[:item]
if raw.is_a? Hash # this is a list of one item
final << raw[:item].inject({}){|x,y| x.merge(y[:key]=>y[:value])}
else
if raw[0][:item].nil? # this is a product info
return raw.inject({}){|x,y| x.merge(y[:key]=>y[:value])}
else # this is a list of many items
raw.each{|result| final << result[:item].inject({}){|x,y| x.merge(y[:key]=>y[:value])}}
end
end
final
end
end
end
And then this:
class Order
def self.get_all_active
activeOrders = MagentoAPI.call 'order.list', :filter => {:status => 'pending'}
end
end
This just returns Savon::HTTP::Error so I'm thinking I'm not formatting the request properly. Does anybody have any experience or insight on this?
Hope this isn't too late (assume it might be), but I created a gem for this with some rudimentary documentation. I'm hoping to finish it up this weekend or next week, but you can take a look at the code and see how I'm creating the filters for Magento. To install, just run:
gem install magento_api_wrapper
To summarize, if you want to use one of the Magento SOAP API simple filters, you can pass a hash with a key and value:
api = MagentoApiWrapper::Sales.new(magento_url: "yourmagentostore.com/index.php", magento_username: "soap_api_username", magento_api_key: "userkey123")
api.order_list(simple_filters: [{key: "status", value: "processing"}, {key: created_at, value: "12/10/2013 12:00" }])
And to use a complex filter, pass a hash with key, operator, and value:
api.order_list(complex_filters: [{key: "status", operator: "eq", value: ["processing", "completed"]}, {key: created_at, operator: "from", value: "12/10/2013" }])
This returns an array of hashes with all your Magento orders.
Specifically, check out the request code: https://github.com/harrisjb/magento_api_wrapper/blob/master/lib/magento_api_wrapper/requests/sales_order_list.rb
While it will be easier to just use the gem, here's how I'm formatting the request prior to passing it to the SavonClient, which finishes the formatting for Magento's SOAP API:
def body
merge_filters!(sales_order_list_hash)
end
def attributes
{ session_id: { "xsi:type" => "xsd:string" },
filters: { "xsi:type" => "ns1:filters" },
}
end
def sales_order_list_hash
{
session_id: self.session_id
}
end
def merge_filters!(sales_order_list_hash)
if !filters_array.empty?
sales_order_list_filters = {
filters: filters_array,
}
sales_order_list_hash.merge!(sales_order_list_filters)
else
sales_order_list_hash
end
end
def filters_array
custom_filters = {}
custom_filters.compare_by_identity
if !simple_filters.nil?
add_simple_filters(custom_filters)
end
if !complex_filters.nil?
add_complex_filters(custom_filters)
end
custom_filters
end
def add_simple_filters(custom_filters)
simple_filters.each do |sfilter|
custom_filters[:attributes!] = {
"filter" => {
"SOAP-ENC:arrayType" => "ns1:associativeEntity[2]",
"xsi:type" => "ns1:associativeArray"
}
}
custom_filters["filter"] = {
item: {
key: sfilter[:key],
value: sfilter[:value], #formatted_timestamp(created_at)
:attributes! => {
key: { "xsi:type" => "xsd:string" },
value: { "xsi:type" => "xsd:string" }
},
},
:attributes! => {
item: { "xsi:type" => "ns1:associativeEntity" },
},
}
end
custom_filters
end
def add_complex_filters(custom_filters)
complex_filters.each do |cfilter|
custom_filters[:attributes!] = {
"complex_filter" => {
"SOAP-ENC:arrayType" => "ns1:complexFilter[2]",
"xsi:type" => "ns1:complexFilterArray"
}
}
custom_filters["complex_filter"] = {
item: {
key: cfilter[:key],
value: {
key: cfilter[:operator],
value: cfilter[:value]
},
:attributes! => {
key: { "xsi:type" => "xsd:string" },
value: { "xsi:type" => "xsd:associativeEntity" }
},
},
:attributes! => {
item: { "xsi:type" => "ns1:complexFilter" },
},
}
end
custom_filters
end
def formatted_timestamp(timestamp)
begin
Time.parse(timestamp).strftime("%Y-%m-%d %H:%M:%S")
rescue MagentoApiWrapper::BadRequest => e
raise "Did you pass date in format YYYY-MM-DD? Error: #{e}"
end
end
def status_array
data[:status_array]
end
def created_at_from
data[:created_at_from]
end
def created_at_to
data[:created_at_to]
end
def last_modified
data[:last_modified]
end
def session_id
data[:session_id]
end
def simple_filters
data[:simple_filters]
end
def complex_filters
data[:complex_filters]
end
I've also got a SavonClient that does some of the configuration for the specific API, here's most of that:
def call
client.call(#request.call_name, message: message_with_attributes, response_parser: :nokogiri)
end
#message_with_attributes are required for some specific formatting when updating Magento via the SOAP API
def message_with_attributes
#request.body.merge!(:attributes! => #request.attributes) unless #request.attributes.empty?
puts "REQUEST: #{#request.inspect}"
return #request.body
end
#configuration of the client is mostly mandatory, however some of these options (like timeout) will be made configurable in the future
#TODO: make timeout configurable
def client
Savon::Client.new do |savon|
savon.ssl_verify_mode :none
savon.wsdl base_url
savon.namespaces namespaces
savon.env_namespace 'SOAP-ENV'
savon.raise_errors false
#savon.namespace_identifier #none
savon.convert_request_keys_to :lower_camelcase
savon.strip_namespaces true
savon.pretty_print_xml true
savon.log log_env
savon.open_timeout 10 #seconds
savon.read_timeout 45 #seconds
end
end
#TODO: make configurable
def log_env
true
end
#correctly format MagentoApiWrapper::Request call_names for SOAP v2
def response_tag_format_lambda
lambda { |key| key.snakecase.downcase }
end
def namespaces
{
'xmlns:SOAP-ENV' => 'http://schemas.xmlsoap.org/soap/envelope/',
'xmlns:ns1' => 'urn:Magento',
'xmlns:xsd' => 'http://www.w3.org/2001/XMLSchema',
'xmlns:xsi' => 'http://www.w3.org/2001/XMLSchema-instance',
'xmlns:SOAP-ENC' => 'http://schemas.xmlsoap.org/soap/encoding/',
'SOAP-ENV:encodingStyle' => 'http://schemas.xmlsoap.org/soap/encoding/'
}
end
#Use MagentoApiWrapper::Api magento_url as endpoint
def base_url
"#{#magento_url}/api/v2_soap?wsdl=1"
end
end
Like I said, it's a work in progress, but I should have pretty good coverage of the Magento API complete in the next couple of weeks. Hope this helps you out! Good luck!

Mixing conditional operators with facets in Elasticsearch

I am trying to match a search query against two fields, as well as filter by facets if selected from dropdowns on the page.
When the user enters keywords it should match if found in two database fields: Title and Description. The dropdowns filter by a status, and a type.
Here is my Tire search configuration:
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { string "title:#{params[:query]}", default_operator: "OR" } if params[:query].present?
should { string "description:#{params[:query]}", default_operator: "OR" } if params[:query].present?
must { term :status_id, params[:status_id] } if params[:status_id].present?
must { term :type_id, params[:type_id] } if params[:type_id].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
facet "status" do
terms :status_id
end
facet "type" do
terms :type_id
end
end
end
Indexing settings:
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3}
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "lowercase",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
indexes :status_id, :type => 'integer'
indexes :type_id, :type => 'integer'
end
end
I originally only had the title and description fields, which was working fine. I am now trying to add the ability to filter by status and type.
What is the proper way to configure this? If status is selected, it should only return records with that status. The same follows for type, and if both are selected.
Any help is appreciated.
It's not that errors occur, but the results no longer filter at all by either keywords or facets:
curl -X GET 'http://localhost:9200/projects/project/_search?load=true&size=25&pretty' -d '{"query":{"bool":{"should":[{"query_string":{"query":"title:test","default_operator":"OR"}},{"query_string":{"query":"description:test","default_operator":"OR"}}],"must":[{"term":{"status_id":{"term":"1"}}},{"term":{"type_id":{"term":"1"}}}]}},"facets":{"status":{"terms":{"field":"status_id","size":10,"all_terms":false}},"type":{"terms":{"field":"type_id","size":10,"all_terms":false}}},"size":25}'
# 2013-08-16 12:08:34:791 [200] (31 msec)
#
# {"took":31,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]},"facets":{"status":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]},"type":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]}}}
If you could create the equivalent of this I think you would get the desired results. (Please excuse the lack of quotes on the JSON keys!)
{
query: {
multi_match: {
query: "test",
fields: ["title", "description"]
}
},
filter: {
and: [
{
term: { status_id: 123 }
},
{
term: { type_id: 456 }
}
]
},
facets: {
type: {
terms: {
field: "type_id",
size: 10
}
},
status: {
terms: {
field: "status_id",
size: 10
}
}
}
}
Update
I don't know tire but will try to write something!
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { match :title params[:query] } if params[:query].present?
should { match :description params[:query] } if params[:query].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
filter :and, { :term => { :status_id => params[:status_id] } } if params[:status_id].present?
{ :term => { :type_id => params[:type_id] } } if params[:type_id].present?
end
end
You will probably have to fix the ruby, but a few things to note. Match queries are the recommended default string search, they are faster than query_string ones (though you have slightly less control). Also

How to filter search by attribute only if it exists using ElasticSearch and Tire?

Right now I wrote
Tire.search INDEX_NAME do
query do
filtered do
query { string term }
filter :or, { missing: { field: :app_id } },
{ terms: { app_id: app_ids } }
end
end
end.results.to_a
Well returning items that either have no app_id or one that matches your terms sounds like a job for an or filter - I'd try
filter :or, [
{:not => {:exists => {:field => :app_id}}},
{:terms => {:app_id => app_ids}}
]

Resources