specifying custom partition and clustering keys when creating vertices with gremlin-python - datastax-enterprise

I have created the following schema using a custom partition and clustering key:
schema.propertyKey('_partition').Text().create()
schema.propertyKey('topic_id').Uuid().create()
schema.vertexLabel('custom_topic')
.partitionKey('_partition').clusteringKey('topic_id').create()
I can now create vertices using gremlin in the following way:
graph.addVertex(label, 'custom_topic', '_partition', 'my_partition', 'topic_id', '60bcae02-f6e5-11e5-9ce9-5e5517507c66')
However, our application is written in python using aiogremlin, and gremlin-python does not provide this graph interface. We can use a traversal to add vertices instead:
g.addV(label, 'custom_topic', '_partition', 'x', 'topic_id', '60bcae02-f6e5-11e5-9ce9-5e5517507c66')
The above works in DSE Studio. However, using gremlin-python it does not work, returning the following error:
aiogremlin.exception.GremlinServerError: 500: 599: Could not locate method: GraphTraversalSource.addV([label, custom_topic, _partition, x, topic_id, 60bcae02-f6e5-11e5-9ce9-5e5517507c66])
DSE reports a DeserialisationError. I tried to pass the keys via properties several different ways:
g.addV('custom_topic').properties('_partition', 'x', 'topic_id', '60bcae02-f6e5-11e5-9ce9-5e5517507c66')
GremlinServerError: 500: 599: Vertices with custom IDs must have their IDs specified on creation.
g.addV('custom_topic').property(T.id, ['_partition', 'x', 'topic_id', '60bcae02-f6e5-11e5-9ce9-5e5517507c66'])
GremlinServerError: 500: 500: Vertex does not support user supplied identifiers
How should I pass these IDs?

Just after posting this question I've found the magic incantation that allows this to work. The key is to specify each property in a separate call:
g.addV('custom_topic').property('_partition', 'x').property('custom_topic', '60bcae02-f6e5-11e5-9ce9-5e5517507c66')
I do not know why properties does not work, but there it is.

Related

Read uncommitted using Firebird FIBPlus components

Using a TpFIBTransaction component, I'm trying to start a READ UNCOMMITTED transaction.
First of all, the TPBMode property has 3 possible values:
tpbDefault
tpbReadCommitted
tpbRepeatableRead
In TpFIBTransaction.StartTransaction I saw that setting tpbReadCommitted forces the following parameters:
write
isc_tpb_nowait
read_committed
rec_version
Using tpbRepeatableRead forces the following parameters instead:
write
isc_tpb_nowait
concurrency
So, it seems the only way to have "custom" transaction parameters is to set the tpbDefault value.
The values allowed for the TrParams property are the following (from fib.pas unit)
TPBConstantNames: array[1..isc_tpb_last_tpb_constant] of String = (
'consistency',
'concurrency',
'shared',
'protected',
'exclusive',
'wait',
'nowait',
'read',
'write',
'lock_read',
'lock_write',
'verb_time',
'commit_time',
'ignore_limbo',
'read_committed',
'autocommit',
'rec_version',
'no_rec_version',
'restart_requests',
'no_auto_undo',
'no_savepoint'
);
I've tried adding the 'read' value only, but it seems it's still unable to read uncommitted data, even if there's not 'read_committed' in TrParams property.
MyTransaction.TrParams.Clear();
MyTransaction.TrParams.Add('read');
Is there some missing value in TPBConstantNames (Something like 'read_uncommitted', if it exists...), or is there another way to setup a Firebird "read uncommitted" transaction?
It is not possible because Firebird does not support read uncomitted isolation level.
You can find the following information in the documentation documentation:
Note
The READ UNCOMMITTED isolation level is a synonym for READ
COMMITTED, and provided only for syntax compatibility. It provides the
exact same semantics as READ COMMITTED, and does not allow you to view
uncommitted changes of other transactions.
and:
The three isolation levels supported in Firebird are:
SNAPSHOT
SNAPSHOT TABLE STABILITY
READ COMMITTED with two specifications (NO RECORD_VERSION and
RECORD_VERSION)

How to Call a Pkg/Procedure Executing an API From a Pkg/Procedure. The API name needs to be Dynamic and Has In and Out Parms

We use APIs, baninst1.PP_DEDUCTION.p_update and baninst1.PP_DEDUCTION.p_create, to maintain our payroll tables of benefit/deduction data. Numerous packages utilize the APIs. We would like to create a package containing the API call that all the existing packages can use and remove the code that is repeated in each package. I tried EXECUTE IMMEDIATE for the purpose of having a dynamic API name. However, I have not been able to get the syntax correct. I’m hoping you will help me.
create or replace PACKAGE BODY "ORBIT"."MM_BENEFITS_COMMON" AS
PROCEDURE PAY_P_EMPLOYEE_BENEFIT_ACTION(pi_benefit_action IN VARCHAR2,
                                                                               pi_pidm                   IN pdrbded.pdrbded_pidm%TYPE,
                                                                               pi_status                  IN pdrdedn.pdrdedn_status%TYPE,
                                                                               pi_bdca_code          IN pdrbded.pdrbded_bdca_code%TYPE,
                                                                               pi_effective_date     IN pdrdedn.pdrdedn_effective_date%TYPE DEFAULT NULL,                                                                               pi_user_id                IN pdrdedn.pdrdedn_user_id%TYPE DEFAULT NULL,                                                                               pi_data_origin          IN pdrdedn.pdrdedn_data_origin%TYPE DEFAULT NULL,
                                                                               po_base_rowid_out  OUT gb_common.internal_record_id_type,
                                                                               po_detail_rowid_out OUT gb_common.internal_record_id_type,
                                                                               pi_amount1              IN pdrdedn.pdrdedn_amount1%TYPE DEFAULT NULL,
                                                                               pi_opt_code1            IN pdrdedn.pdrdedn_opt_code1%TYPE DEFAULT NULL) IS
BEGIN
--Call the API for p_create or p_update.
baninst1.PP_DEDUCTION.pi_benefit_action(p_pidm                  => pi_pidm,
                                                                       p_status                 => pi_status,
                                                                       p_bdca_code          => pi_bdca_code,
                                                                       p_effective_date     => CASE
                                                                                                                 WHEN pi_benefit_action 'p_create' THEN
                                                                                                                       TRUNC(pi_begin_date)
                                                                                                                 ELSE
                                                                                                                       TRUNC(pi_effective_date)
                                                                                                             END,
                                                                      p_user_id                =>   pi_user_id,
                                                                      p_data_origin          =>   pi_data_origin,
                                                                      p_base_rowid_out   =>   po_base_rowid_out,
                                                                      p_detail_rowid_out  =>  po_detail_rowid_out,                                                                      p_amount1              =>  pi_amount1,
                                                                      p_opt_code1            => CASE
                                                                                                                 WHEN LENGTH(pi_opt_code1) = 1 THEN
                                                                                                                        '0' || pi_opt_code1
                                                                                                                 ELSE pi_opt_code1
                                                                                                            END);
END PAY_P_EMPLOYEE_BENEFIT_ACTION;
END MM_BENEFITS_COMMON;
create or replace PACKAGE BODY "ORBIT"."MM_BENEFIT_TEST" AS
PROCEDURE PAY_P_MM_BENEFIT_TEST IS
lv_base_rowid_out    gb_common.internal_record_id_type;
lv_detail_rowid_out   gb_common.internal_record_id_type;
BEGIN
--Pass data to the common benefits package for the api call
MM_BENEFITS_COMMON.PAY_P_EMPLOYEE_BENEFIT_ACTION('p_update', 9999999, 'A', 'VI1', '01-JAN-2022', 'MM_Test',     'MM_TEST', lv_base_rowid_out, lv_detail_rowid_out, 25.82, NULL);
END PAY_P_MM_BENEFIT_TEST;
END MM_BENEFIT_TEST;
I'm not sure what's bothering you, actually. You did post some code, but - I don't know what it represents.
Let's see what you said:
"We use APIs, baninst1.PP_DEDUCTION.p_update and baninst1.PP_DEDUCTION.p_create, to maintain our payroll tables of benefit/deduction data."
OK
"Numerous packages utilize the APIs."
it means that there are many packages and they call those p_update and p_create procedures; that's also OK
"We would like to create a package containing the API call that all the existing packages can use and remove the code that is repeated in each package."
that would be a new package; you'd cut that piece of code from all of your packages and paste it into a new one.
OK, makes sense. Instead of all that code (in every package), you'd put call to newly created procedures (that reside in a newly created package)
"I tried EXECUTE IMMEDIATE for the purpose of having a dynamic API name. However, I have not been able to get the syntax correct."
why dynamic SQL? There's nothing dynamic here. Instead of dozens of lines of code you currently have (that do something), you'd put one line - the one that calls that newly created procedure (and pass parameters)
From my point of view, there's nothing unusual in what you want to do and I can't imagine what problems you could have in doing it; it's pretty much straightforward.

Stored procedures in gremlin CosmosDB

I'm new to gremlin and cosmos DB and was trying to use Stored Procedure in cosmos DB gremlin API.
I started with Quick-start-nodejs doc for creating a node.js app, connected with CosmosDB gremlin API. Now I want to use Stored Procedure in that app.
I found only single doc for stored procedures in cosmos DB, and that's only for Document DB (in SQL). I didn't found any doc related to stored procedure in gremlin.
Can anyone guide me, how to do that?
Thanks in advance.
I had the same problem as you. And found that Cosmos DB in Gremlin or Graph mode does not support stored procedures. You can create them from UI but because you can not use any Gremlin query or have trigger - they are useless. Also there is no documentation on it.
I have found a post from March 2019 that stored procedures are on the roadmap for Germlin
https://feedback.azure.com/forums/263030-azure-cosmos-db/suggestions/20115355-gremlin-queries-from-stored-procedures
Personally, for my use case I am considering use of Neo4j instead of Cosmos because of the lack of stored procedures
What's your use case?
Gremlin is a language for traversing graph. Gremlin has no knowledge of CosmosDB stored procedure and hence you can't really execute stored procedure via gremlin.
However, CosmosDB is multi-model. You can talk to it via gremlin as well as native DocumentDB API.
You should look into how to execute stored procedure via DocumentDB API.
Based on your comment on the first answer to the question: "Actually I want to add some edges every time a new vertex is created. For example, whenever a vertex with the label EMPLOYEE is created, an edge to the vertex COMPANY must be automatically created." here, you can look into TinkerPop's EventStrategy.
EDIT:
Adding essential parts from the link above in case the link changes:
The purpose of the EventStrategy is to raise events to one or more MutationListener objects as changes to the underlying Graph occur within a Traversal. Such a strategy is useful for logging changes, triggering certain actions based on change, or any application that needs notification of some mutating operation during a Traversal. If the transaction is rolled back, the event queue is reset.
The following events are raised to the MutationListener:
New vertex
New edge
Vertex property changed
Edge property changed
Vertex property removed
Edge property removed
Vertex removed
Edge removed
To start processing events from a Traversal first implement the MutationListener interface. An example of this implementation is the ConsoleMutationListener which writes output to the console for each event. The following console session displays the basic usage:
gremlin> import org.apache.tinkerpop.gremlin.process.traversal.step.util.event.*
==>org.apache.tinkerpop.gremlin.structure.*, org.apache.tinkerpop.gremlin.structure.util.*, org.apache.tinkerpop.gremlin.process.traversal.*, org.apache.tinkerpop.gremlin.process.traversal.step.*, org.apache.tinkerpop.gremlin.process.remote.*, org.apache.tinkerpop.gremlin.structure.util.empty.*, org.apache.tinkerpop.gremlin.structure.io.*, org.apache.tinkerpop.gremlin.structure.io.graphml.*, org.apache.tinkerpop.gremlin.structure.io.graphson.*, org.apache.tinkerpop.gremlin.structure.io.gryo.*, org.apache.commons.configuration.*, org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.*, org.apache.tinkerpop.gremlin.process.traversal.strategy.optimization.*, org.apache.tinkerpop.gremlin.process.traversal.strategy.finalization.*, org.apache.tinkerpop.gremlin.process.traversal.strategy.verification.*, org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.*, org.apache.tinkerpop.gremlin.process.traversal.util.*, org.apache.tinkerpop.gremlin.process.computer.*, org.apache.tinkerpop.gremlin.process.computer.bulkdumping.*, org.apache.tinkerpop.gremlin.process.computer.bulkloading.*, org.apache.tinkerpop.gremlin.process.computer.clustering.peerpressure.*, org.apache.tinkerpop.gremlin.process.computer.traversal.*, org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.*, org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.optimization.*, org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.*, org.apache.tinkerpop.gremlin.util.*, org.apache.tinkerpop.gremlin.util.iterator.*, static org.apache.tinkerpop.gremlin.structure.io.IoCore.*, static org.apache.tinkerpop.gremlin.process.traversal.P.*, static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.*, static org.apache.tinkerpop.gremlin.process.computer.Computer.*, static org.apache.tinkerpop.gremlin.util.TimeUtil.*, static org.apache.tinkerpop.gremlin.process.traversal.SackFunctions.Barrier.*, static org.apache.tinkerpop.gremlin.structure.VertexProperty.Cardinality.*, static org.apache.tinkerpop.gremlin.structure.Column.*, static org.apache.tinkerpop.gremlin.structure.Direction.*, static org.apache.tinkerpop.gremlin.process.traversal.Operator.*, static org.apache.tinkerpop.gremlin.process.traversal.Order.*, static org.apache.tinkerpop.gremlin.process.traversal.Pop.*, static org.apache.tinkerpop.gremlin.process.traversal.Scope.*, static org.apache.tinkerpop.gremlin.structure.T.*, static org.apache.tinkerpop.gremlin.process.traversal.step.TraversalOptionParent.Pick.*, org.apache.tinkerpop.gremlin.driver.*, org.apache.tinkerpop.gremlin.driver.exception.*, org.apache.tinkerpop.gremlin.driver.message.*, org.apache.tinkerpop.gremlin.driver.ser.*, org.apache.tinkerpop.gremlin.driver.remote.*, groovyx.gbench.*, groovyx.gprof.*, static groovyx.gprof.ProfileStaticExtension.*, org.apache.tinkerpop.gremlin.giraph.process.computer.*, org.apache.hadoop.conf.*, org.apache.hadoop.hdfs.*, org.apache.hadoop.fs.*, org.apache.hadoop.util.*, org.apache.hadoop.io.*, org.apache.hadoop.io.compress.*, org.apache.hadoop.mapreduce.lib.input.*, org.apache.hadoop.mapreduce.lib.output.*, org.apache.tinkerpop.gremlin.hadoop.*, org.apache.tinkerpop.gremlin.hadoop.structure.*, org.apache.tinkerpop.gremlin.hadoop.structure.util.*, org.apache.tinkerpop.gremlin.hadoop.structure.io.*, org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.*, org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.*, org.apache.tinkerpop.gremlin.hadoop.structure.io.script.*, org.apache.tinkerpop.gremlin.hadoop.process.computer.mapreduce.*, org.apache.tinkerpop.gremlin.groovy.jsr223.dsl.credential.*, static org.apache.tinkerpop.gremlin.groovy.jsr223.dsl.credential.CredentialGraph.*, org.apache.tinkerpop.gremlin.neo4j.structure.*, static org.apache.tinkerpop.gremlin.neo4j.process.traversal.LabelP.*, org.apache.tinkerpop.gremlin.spark.process.computer.*, org.apache.tinkerpop.gremlin.spark.structure.*, org.apache.tinkerpop.gremlin.spark.structure.io.*, org.apache.tinkerpop.gremlin.tinkergraph.structure.*, org.apache.tinkerpop.gremlin.tinkergraph.process.computer.*, org.apache.tinkerpop.gremlin.process.traversal.step.util.event.*
gremlin> graph = TinkerFactory.createModern()
==>tinkergraph[vertices:6 edges:6]
gremlin> l = new ConsoleMutationListener(graph)
==>MutationListener[tinkergraph[vertices:6 edges:6]]
gremlin> strategy = EventStrategy.build().addListener(l).create()
==>EventStrategy
gremlin> g = graph.traversal().withStrategies(strategy)
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g.addV().property('name','stephen')
Vertex [v[13]] added to graph [tinkergraph[vertices:7 edges:6]]
==>v[13]
gremlin> g.E().drop()
Edge [e[7][1-knows->2]] removed from graph [tinkergraph[vertices:7 edges:6]]
Edge [e[8][1-knows->4]] removed from graph [tinkergraph[vertices:7 edges:5]]
Edge [e[9][1-created->3]] removed from graph [tinkergraph[vertices:7 edges:4]]
Edge [e[10][4-created->5]] removed from graph [tinkergraph[vertices:7 edges:3]]
Edge [e[11][4-created->3]] removed from graph [tinkergraph[vertices:7 edges:2]]
Edge [e[12][6-created->3]] removed from graph [tinkergraph[vertices:7 edges:1]]
By default, the EventStrategy is configured with an EventQueue that raises events as they occur within execution of a Step. As such, the final line of Gremlin execution that drops all edges shows a bit of an inconsistent count, where the removed edge count is accounted for after the event is raised. The strategy can also be configured with a TransactionalEventQueue that captures the changes within a transaction and does not allow them to fire until the transaction is committed.
WARNING
EventStrategy is not meant for usage in tracking global mutations across separate processes. In other words, a mutation in one JVM process is not raised as an event in a different JVM process. In addition, events are not raised when mutations occur outside of the Traversal context.

erlang connecting to tinkerpop via REST

In the Tinkerpop or Titan documentation, all operations are based on a sample graph. How to creat a new empty graph to work on?
I am programming in erlang connecting to Tinkergraph, planned to use Titan later in production. There is no erlang driver for both so I am connecting by REST. It is easy to read from graph, but if I want to read from user's input then write into the graph, for example, to create a person named teddy:
screenshot 1
I got those errors. What is the correct way?
Thank you.
Update: For following situation:
23> Newperson=terry.
terry
24> Newperson.
terry
If I want to add this terry, below two will not work. What's the correct way to do it?
screenshot 2
1
TitanGraph titanGraph = TitanFactory.open(config); will open a titan graph without the sample data.
If you have already commited the sample data to your keyspace then you can just change the keyspace defined in your config file.
For example if you are using a cassandra backend you would change storage.cassandra.keyspace=xxxxxx .
You can also clear any keyspace using TitanCleanup.clear(graph);
2
As for the error you are seeing. It looks like you are trying to label your vertex incorrectly. I posted the following and it worked:
{
"gremlin" : "g.addV(label, x).property(y,z)",
"bindings" :
{
"x" : "person",
"y" : "name",
"z" : "Teddy"
}
}
A final note, when you start using Titan 1.0.0 make sure you checkout this section of the tinkerpop docs. Especially make sure to change the channel in the gremlin-server.yaml config to:
channelizer: com.tinkerpop.gremlin.server.channel.HttpChannelizer
Answer to my own question: construct a Body by lists:concat() or ++, then post

Riak Erlang map/reduce returns {error,notfound}

I've running the following Riak map phase:
-module(delete_map_function).
-export([get_keys/3]).
%Returns bucket and key pairs from a map phase
get_keys(Value,_Keydata,_Arg) ->
[[riak_object:bucket(Value),riak_object:key(Value)]].
And the following Riak reduce phase: http://contrib.basho.com/delete_keys.html
I keep getting this error message:
{"phase":0,"error":"function_clause","input":"{{error,notfound},{<<\"my_bucket\">>,<<\"item_key\">>},undefined}","type":"error","stack":"[{riak_object,bucket,[{error,notfound}],[{file,\"src/riak_object.erl\"},{line,251}]},{delete_map_function,get_keys,3,[{file,\"delete_map_function.erl\"},{line,7}]},{riak_kv_mrc_map,map,3,[{file,\"src/riak_kv_mrc_map.erl\"},{line,164}]},{riak_kv_mrc_map,process,3,[{file,\"src/riak_kv_mrc_map.erl\"},{line,140}]},{riak_pipe_vnode_worker,process_input,3,[{file,\"src/riak_pipe_vnode_worker.erl\"},{line,444}]},{riak_pipe_vnode_worker,wait_for_input,2,[{file,\"src/riak_pipe_vnode_worker.erl\"},{line,376}]},{gen_fsm,...},...]"}
I'm running the job via Java:
MapReduceResult mapReduceResult = RiakUtils.getPBClient().mapReduce(iq)
.addMapPhase(new NamedErlangFunction("delete_map_function", "get_keys"))
.addReducePhase(new NamedErlangFunction("delete_reduce_function", "delete"))
.execute();
I've read somewhere that I should use the filter_notfound argument in the Map phase, but I still keep getting the error even after adding it:
MapReduceResult mapReduceResult = RiakUtils.getPBClient().mapReduce(iq)
.addMapPhase(new NamedErlangFunction("delete_map_function", "get_keys"), "filter_notfound")
.addReducePhase(new NamedErlangFunction("delete_reduce_function", "delete"))
.execute();
I'm running Riak 1.3 and using the Riak Java client v1.1.0
First. I think it is not efficient way to delete keys via map/reduce phases. if you got keys list and feed it to map phase riak will first read all objects and then give it to your functions. So if you only need to delete object, better do it without read.
Second. All your map/reduce functions should be written with following exceptions:
instead of Value, you can get {error, notfound} because of eventual nature of riak.
also you can get deleted riak object as Value. You can know that object was deleted by special flag dict:is_key(<<"X-Riak-Deleted">>, riak_object:get_metadata(RiakObj)):
Third. To fix your error you should filter notfound keys from the list:
get_keys({error, notfound},_Keydata,_Arg) ->
[];
get_keys(Value,_Keydata,_Arg) ->
[[riak_object:bucket(Value),riak_object:key(Value)]].

Resources