DSE Graph: traversals that use both secondary and search indexes, why do they hang? - datastax-enterprise

It seems that gremlin traversals which use both secondary and search indexes in the same traversal hang. Why is that?
My understanding from the DSE Graph docs on indexing is that the most appropriate index for low-cardinality properties is a secondary index. I have a model with a 'type' property such that there are a limited number of possible types; so when I needed an index I used a secondary index.
But it seems that it's not possible to use both secondary and search indexes in the same traversal, as demonstrated by the following example:
gremlin> system.graph('example').create()
==>null
gremlin> :remote config alias g example.g
==>g=example.g
gremlin> schema.vertexLabel('item').create()
==>null
gremlin> schema.propertyKey('type').Text().single().create()
==>null
gremlin> schema.propertyKey('description').Text().single().create()
==>null
gremlin> schema.vertexLabel('item').properties('type').add()
==>null
gremlin> schema.vertexLabel('item').properties('description').add()
==>null
gremlin> schema.vertexLabel('item').index('byType').secondary().by('type').add()
==>null
gremlin> schema.vertexLabel('item').index('search').search().by('description').add()
==>null
gremlin> graph.addVertex(label, 'item', 'type', 'A', 'description', 'first item is orange')
==>v[{~label=item, community_id=96406144, member_id=0}]
gremlin> graph.addVertex(label, 'item', 'type', 'A', 'description', 'second item is blue')
==>v[{~label=item, community_id=2076720128, member_id=0}]
gremlin> graph.addVertex(label, 'item', 'type', 'B', 'description', 'third item is green')
==>v[{~label=item, community_id=51027072, member_id=0}]
gremlin> g.V().hasLabel('item').has('type', 'A')
==>v[{~label=item, community_id=2076720128, member_id=0}]
==>v[{~label=item, community_id=96406144, member_id=0}]
gremlin> g.V().hasLabel('item').has('description', Search.token('blue'))
==>v[{~label=item, community_id=2076720128, member_id=0}]
gremlin> g.V().hasLabel('item').has('type', 'A').has('description', Search.token('blue'))
On the last traversal, the server logs the following statement in /var/log/cassandra/system.log:
WARN [gremlin-server-worker-1] 2016-09-15 14:26:49,759 GremlinExecutor.java:267 - Timing out script - g.V().hasLabel('item').has('type', 'A').has('description', Search.token('blue')) - in thread [gremlin-server-worker-1]
And the console freezes up completely, and doesn't even respond to SIGTERM.

This is resolved by DSE v5.0.3.

Related

Mongoid: inconsistent results on Query when using $elemMatch for nested arrays

Context
puts [
"#{RUBY_ENGINE} #{RUBY_VERSION}",
"Rails gem: #{Rails.gem_version.version}",
"MongoDB #{Mongoid.default_client.command(buildInfo: 1).first[:version]}",
"Mongoid gem: #{Mongoid::VERSION}"
].join("\n")
ruby 2.7.6
Rails gem: 5.2.4.5
MongoDB 4.4.18
Mongoid gem: 7.5.1
Scenario
To this relationships: Foo#bars, Bar#bazes, these are the cases aimed for:
Identify Bar documents where any label of bazes matches cheese.
Identify Bar documents where any id of bazes matches a target id.
Identify Bar documents where bazes have any of an array of ids.
Although I could work out (1), cases (2) and (3) remain a mystery. Below an example, where in Queries and Results there is some example.
Example
Data Model
class Baz
include Mongoid::Document
embedded_in :bar
field :label, type: String
end
class Bar
include Mongoid::Document
embedded_in :foo
embeds_many :bazes, class_name: "Baz"
end
class Foo
include Mongoid::Document
field :name, type: String
embeds_many :bars, class_name: "Bar"
end
Data Sample
foo = Foo.create!({
name: "foo",
bars: [
{bazes: [{label: "table"}, {label: "chair"}]},
{bazes: [{label: "bread"}, {label: "cheese"}]},
{bazes: [{label: "up"}, {label: "down"}]},
{bazes: [{label: "high"}]},
{bazes: [{label: "low"}]}
]
})
Queries and Results
Select the target for cases (1) and (2):
baz = foo.bars[1].bazes.last
baz.label
#=> "cheese"
baz.id
#=> BSON::ObjectId('638d3b824b9f26049b42ac0a')
Case 1
foo.bars.where(bazes: {"$elemMatch": {label: baz.label}}).count
#=> 1
It seems to be working properly. Doing some double check:
bazes = foo.bars.where(bazes: {"$elemMatch": {label: baz.label}}).pluck(:bazes).first
found = bazes.where(label: baz.label).first
#=> #<Baz _id: 638d3b824b9f26049b42ac0a, label: "cheese">
found == baz
#=> true
It certainly worked.
Case 2
Using exactly the same syntax as in Case (1), but matching the id field instead:
foo.bars.where(bazes: {"$elemMatch": {id: baz.id}}).count
#=> 0
It does not work.
At this point I am quite lost. How would a field like label work but the id wouldn't?
Case 3
The below is what I guess it should look like. It doesn't work either, although it makes sense, given that the base case (2) above fails in the first place...
foo.bars.where(bazes: {"$elemMatch": {:id.in => [baz.id]}}).count
#=> 0

How to combine two joins.where in rails

Help to join the query in one large, well, or grammatically, so that for each company there are dot applications in which
Here are 2 separately working query strings.
DotApplication.all.joins(:current_stage_event).where(current_stage_event: {event: "prospect"}).joins('LEFT JOIN tasks on tasks.subject_id = dot_applications.id').where(tasks: {id: nil})
Company.all.joins(:users).where(users: { notification_for_active_prospects: true})
What i want:
Company.each do |company|
# For each company(with joins,where above), find all dot_applications as in the examples above
end
The association between Company and DotApplication looks like this:
# app/models/company.rb
class Comany < ApplicationRecord
has_many :dot_aplications
end
# app/models/dot_application.rb
class DotApplication < ApplicationRecord
belongs_to :company
end
You can join on nested associations like this joins(company: :users)
In your case this might do the job
DotApplication.joins(company: :user).where(users: { notification_for_active_prospects: true})
Or full version
DotApplication.all.joins(:current_stage_event, company: :user).where(current_stage_event: {event: "prospect"}).joins('LEFT JOIN tasks on tasks.subject_id = dot_applications.id').where(tasks: {id: nil}).where(users: { notification_for_active_prospects: true})
But I really don't recommand you to do this at it is totally unmaintainable
You can read this post of you want to know more about joins https://www.learneroo.com/modules/137/nodes/768
Company.joins(:dot_applications)
.where(dot_appplications: DotApplication.joins(:current_stage_event)
.where(current_stage_event: {event: "prospect"})
.joins('LEFT JOIN tasks on tasks.subject_id = dot_applications.id')
.where(tasks: {id: nil}))
.joins(:users)
.where(users: { notification_for_active_prospects: true})
This should find all the companies meeting your requirements for the DotApplication and then query those companies for the User requirements

Intersection of 2 arrays of hashes by hash value (such as id)

In my rails app, I have a user model and a linkedin_connection model. Linkedin_connection belongs to user and user has_many linkedin_connections.
What's the best way to create a crossover array of connections between user1 and user2?
============== EDIT ============== EDIT ============== EDIT ==============
I realized that this is sort of a different problem than I originally thought. I have 2 arrays of hashes. Each hash has an id element. How can I find the intersection of two hashes by their ids?
Example
user1.linkedin_connections => [{id: "123xyz", name: "John Doe"}, {id: "789abc", name: "Alexander Supertramp"}]
user2.linkedin_connections => [{id: "456ijk", name: "Darth Vader"}, {id: "123xyz", name: "John Doe"}]
cross_connections => [{id: "123xyz", name: "John Doe"}]
How can I calculate "cross_connections?"
Thanks!
What you want is the intersection of the two arrays. In ruby, that's easy, using the & operator:
crossover_connections = user1.linkedin_connections.to_a & user2.linkedin_connections.to_a

idiomatic map join in clojure?

I hava a data structure that looks like this:
(def conf
{ :devices [{:alias "OSC Sender",
:name "OSC Sender",
:ins [{:name "xpos", :type :int, :mutable true}]},
{:alias "const2", :name "const",
:outs [{:name "out", :type :int}]}],
:connections {"const2.out" "OSC Sender.xpos"},
:layout [{:alias "const2",
:x 72.12447405329594,
:y 99.88499298737729},
{:alias "tick",
:x 82.5732819074334,
:y 133.91374474053296},
{:alias "OSC Sender",
:x 185.17741935483872,
:y 113.90322580645162}]})
I would like to join maps in :devices and :layout by key (specifically :alias) to enrich the devices with layout information.
Right now I cobbled the following solution:
(map (partial reduce merge) (vals (group-by :alias (concat (:devices conf) (:layout conf)))))
Is that an idiomatic join or is something else preferable?
Cheers
You can use the join function from the clojure.set namespace:
(clojure.set/join (conf :devices) (conf :layout) {:alias :alias})
Note that the return value is a set. Omitting the final argument results in a natural join; see (doc clojure.set/join) for details.

Help with creating an array

I have a categories model.
attr_accessible :name, :parent, :tags, :active, :image
Sample data will be something like
"Bakery",null,"bakehouse, baker, bakeries, bakers, bakery, bakeshop, biscuit factory, boulangerie, bread bakery, bread, cake bakery, cake topper, cake toppers, cake, cakes, confectionery, cook shop, pastry kitchen, pastry shop, pastry, patisserie, wedding cake", 1, null
"Beauty & Makeup",null,"makeup, ", 1, null
Doing #categories = Category.all gives me all categories with all the fields, now I want to create an array with the tags and the category name. The array data should be like
[["Bakery", "bakehouse"]
["Bakery", "baker"]
["Bakery", "bakeries"]
["Bakery", "bakers"]
["Bakery", "bakery"]
.....
["Beauty & Makeup", "makeup"]]
how can this be done?
#categories.map {|cat| cat.tags.split(/, /).map {|tag| [cat.name, tag]}} should do the trick.

Resources