Platform:
Neo4j - Graph Database Kernel 2.0.0-M03 (server)
jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on Java HotSpot(TM) 64-Bit Server VM 1.7.0_25-b15 [linux-amd64]
neography (1.1.1) for interaction with the neo4j REST api
I have a series of product nodes, with indexed catalogue number properties already stored in neo4j, and I'm attempting to batch insert SKUs that are children / variations of those products, with a relationship from the SKU -> the parent Product catalogue number
I have the following code:
#items is an array of skus
items.each do |item|
commands = []
#get the node we want to create a relationship to
commands << [:get_node_index, PRODUCT_CATALOG_INDEX, :catalog_number, item[:catalog_number]]
commands << [:create_node, {sku: item[:sku]}]
commands << [:add_node_to_index, PRODUCT_SKU_INDEX, :sku, item[:itemnumber], '{1}']
commands << [:create_relationship, "product_variation_of", '{1}', '{0}']
#neo.batch *commands
end
This returns back the error (full stack trace at the bottom)
, "body"=>{"message"=>"For input string: \"{0}\""
, "exception"=>"BadInputException"
, "fullname"=>"org.neo4j.server.rest.repr.BadInputException"
Clearly I'm doing something wrong here, but I can't seem to see what it is. I'm wondering if there is something wrong with: [:get_node_index, PRODUCT_CATALOG_INDEX, :catalog_number, item[:catalog_number]], that isn't allowing the node reference of {0} ? I'm honestly not sure.
Any help is appreciated!
Full stack trace, with ,'s replaced with \n
E
, [2013-07-25T19:18:37.080000 #29182] ERROR -- : 400 error: [{"id"=>0
, "from"=>"/index/node/product_catalog_number/catalog_number/1234"
, "body"=>[{"extensions"=>{}
, "outgoing_relationships"=>"http://dev.local:7474/db/data/node/531/relationships/out"
, "labels"=>"http://dev.local:7474/db/data/node/531/labels"
, "all_typed_relationships"=>"http://dev.local:7474/db/data/node/531/relationships/all/{-list|&|types}"
, "traverse"=>"http://dev.local:7474/db/data/node/531/traverse/{returnType}"
, "self"=>"http://dev.local:7474/db/data/node/531"
, "property"=>"http://dev.local:7474/db/data/node/531/properties/{key}"
, "properties"=>"http://dev.local:7474/db/data/node/531/properties"
, "outgoing_typed_relationships"=>"http://dev.local:7474/db/data/node/531/relationships/out/{-list|&|types}"
, "incoming_relationships"=>"http://dev.local:7474/db/data/node/531/relationships/in"
, "create_relationship"=>"http://dev.local:7474/db/data/node/531/relationships"
, "paged_traverse"=>"http://dev.local:7474/db/data/node/531/paged/traverse/{returnType}{?pageSize
,leaseTime}"
, "all_relationships"=>"http://dev.local:7474/db/data/node/531/relationships/all"
, "incoming_typed_relationships"=>"http://dev.local:7474/db/data/node/531/relationships/in/{-list|&|types}"
, "data"=>{"title"=>"MY PRODUCT"
, "catalog_number"=>"1234"}
, "indexed"=>"http://dev.local:7474/db/data/index/node/product_catalog_number/catalog_number/1234/531"}]
, "status"=>200}
, {"id"=>1
, "from"=>"/node"
, "body"=>{"extensions"=>{}
, "outgoing_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/out"
, "labels"=>"http://dev.local:7474/db/data/node/1239/labels"
, "all_typed_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/all/{-list|&|types}"
, "traverse"=>"http://dev.local:7474/db/data/node/1239/traverse/{returnType}"
, "self"=>"http://dev.local:7474/db/data/node/1239"
, "property"=>"http://dev.local:7474/db/data/node/1239/properties/{key}"
, "properties"=>"http://dev.local:7474/db/data/node/1239/properties"
, "outgoing_typed_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/out/{-list|&|types}"
, "incoming_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/in"
, "create_relationship"=>"http://dev.local:7474/db/data/node/1239/relationships"
, "paged_traverse"=>"http://dev.local:7474/db/data/node/1239/paged/traverse/{returnType}{?pageSize
,leaseTime}"
, "all_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/all"
, "incoming_typed_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/in/{-list|&|types}"
, "data"=>{"sku"=>"0320442A5"}}
, "location"=>"http://dev.local:7474/db/data/node/1239"
, "status"=>201}
, {"id"=>2
, "from"=>"/index/node/product_sku"
, "body"=>{"extensions"=>{}
, "outgoing_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/out"
, "labels"=>"http://dev.local:7474/db/data/node/1239/labels"
, "all_typed_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/all/{-list|&|types}"
, "traverse"=>"http://dev.local:7474/db/data/node/1239/traverse/{returnType}"
, "self"=>"http://dev.local:7474/db/data/node/1239"
, "property"=>"http://dev.local:7474/db/data/node/1239/properties/{key}"
, "properties"=>"http://dev.local:7474/db/data/node/1239/properties"
, "outgoing_typed_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/out/{-list|&|types}"
, "incoming_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/in"
, "create_relationship"=>"http://dev.local:7474/db/data/node/1239/relationships"
, "paged_traverse"=>"http://dev.local:7474/db/data/node/1239/paged/traverse/{returnType}{?pageSize
,leaseTime}"
, "all_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/all"
, "incoming_typed_relationships"=>"http://dev.local:7474/db/data/node/1239/relationships/in/{-list|&|types}"
, "data"=>{"sku"=>"0320442A5"}
, "indexed"=>"http://dev.local:7474/db/data/index/node/product_sku/sku/0320442A5/1239"}
, "location"=>"http://dev.local:7474/db/data/index/node/product_sku/sku/0320442A5/1239"
, "status"=>201}
, {"id"=>3
, "from"=>"http://dev.local:7474/db/data/node/1239/relationships"
, "body"=>{"message"=>"For input string: \"{0}\""
, "exception"=>"BadInputException"
, "fullname"=>"org.neo4j.server.rest.repr.BadInputException"
, "stacktrace"=>["org.neo4j.server.rest.web.RestfulGraphDatabase.extractNodeId(RestfulGraphDatabase.java:199)"
, "org.neo4j.server.rest.web.RestfulGraphDatabase.createRelationship(RestfulGraphDatabase.java:564)"
, "java.lang.reflect.Method.invoke(Method.java:606)"
, "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"
, "org.neo4j.server.web.Jetty6WebServer.invokeDirectly(Jetty6WebServer.java:291)"
, "org.neo4j.server.rest.web.StreamingBatchOperations.invoke(StreamingBatchOperations.java:66)"
, "org.neo4j.server.rest.batch.BatchOperations.performRequest(BatchOperations.java:188)"
, "org.neo4j.server.rest.batch.BatchOperations.parseAndPerform(BatchOperations.java:159)"
, "org.neo4j.server.rest.web.StreamingBatchOperations.readAndExecuteOperations(StreamingBatchOperations.java:54)"
, "org.neo4j.server.rest.web.BatchOperationService$1.write(BatchOperationService.java:89)"
, "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]
, "cause"=>{"message"=>"For input string: \"{0}\""
, "exception"=>"NumberFormatException"
, "fullname"=>"java.lang.NumberFormatException"
, "stacktrace"=>["java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)"
, "java.lang.Long.parseLong(Long.java:441)"
, "java.lang.Long.parseLong(Long.java:483)"
, "org.neo4j.server.rest.web.RestfulGraphDatabase.extractNodeId(RestfulGraphDatabase.java:195)"
, "org.neo4j.server.rest.web.RestfulGraphDatabase.createRelationship(RestfulGraphDatabase.java:564)"
, "java.lang.reflect.Method.invoke(Method.java:606)"
, "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"
, "org.neo4j.server.web.Jetty6WebServer.invokeDirectly(Jetty6WebServer.java:291)"
, "org.neo4j.server.rest.web.StreamingBatchOperations.invoke(StreamingBatchOperations.java:66)"
, "org.neo4j.server.rest.batch.BatchOperations.performRequest(BatchOperations.java:188)"
, "org.neo4j.server.rest.batch.BatchOperations.parseAndPerform(BatchOperations.java:159)"
, "org.neo4j.server.rest.web.StreamingBatchOperations.readAndExecuteOperations(StreamingBatchOperations.java:54)"
, "org.neo4j.server.rest.web.BatchOperationService$1.write(BatchOperationService.java:89)"
, "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]}}
, "status"=>400}]
You have it right. You can't use a result from an index as a single node reference because you can get a collection from that index lookup. (Say you index first name, and you get all the people named bob, and you get 10 back... problem.)
So run the index query, get the first node id from the array that comes back and then pass that in to your your batch as proper id.
The identifier {0} etc. can right now only refer to location headers which are not set by index get commands as they can return an arbitrary number of nodes.
AFAIK the get-or-create command sets the correct location header.
POST http://localhost:7474/db/data/index/node/PRODUCT_CATALOG_INDEX?uniqueness=get_or_create \
{"key" : "catalog_number","value" : item[:catalog_number],\
"properties" : {"catalog_number" : item[:catalog_number]}}
Related
I have been following along with Felipe Hoffa's blog post "Easy pivot() in BigQuery" (https://towardsdatascience.com/easy-pivot-in-bigquery-one-step-5a1f13c6c710) and I've been able to successfully call his procedure and replicate his example calculations. However, because the data I'm ultimately interested in are hosted in the EU, I can't call his procedure verbatim and have been unsuccessfully trying to create and run a copy of the code into my own personal BigQuery project folder as a result.
As best I can tell, the steps involved are.
Copy the code here https://github.com/fhoffa/code_snippets/blob/5163b921398ee29a8010c164a17af05268ac8639/util/pivot.sql
Update the project ID and dataset (e.g. swap out every instance of "`fhoffa.x." with my BigQuery info "blah.matt.") and create the stored procedure in my own BigQuery account
Run the code, adjusting for the new location.
Something like:
CALL `*blah*.matt.pivot`(
'bigquery-public-data.iowa_liquor_sales.sales' # source table
, 'blah.matt.test2' # destination table
, ['date'] # row_ids
, 'store_number' # pivot_col_name
, 'sale_dollars' # pivot_col_value
, 5 # max_columns
, 'SUM' # aggregation
, '' # optional_limit
);
Unfortunately, when I do this, I encounter the following error.
Invalid EXECUTE IMMEDIATE sql string SELECT STRING_AGG(' SUM(IF('||#pivot_col_name||'="'||x.value||'", '||#pivot_col_value||', null)) e_'||blah.matt.normalize_col_name(x.value)) FROM UNNEST(( SELECT APPROX_TOP_COUNT(store_number, #max_columns) FROM bigquery-public-data.iowa_liquor_sales.sales)) x, Syntax error: Expected ")" but got identifier "matt" at [blah.matt.pivot:5:5]
Can anyone please advise? I'm a novice at Big Query, and can't tell where I've gone off the rails. Apologies if this is a super basic question.
Thank you very much for your time and help!
Regards,
Matt
Most likely your problem is related to not properly referencing your project.dataset or just simply having typo, etc. - to be on safe side do as below (just copy paste from below)
CREATE OR REPLACE FUNCTION
`blah.matt.normalize_col_name`(col_name STRING) AS (
REGEXP_REPLACE(col_name,r'[/+#|]', '_')
);
CREATE OR REPLACE PROCEDURE `blah.matt.pivot`(
table_name STRING
, destination_table STRING
, row_ids ARRAY<STRING>
, pivot_col_name STRING
, pivot_col_value STRING
, max_columns INT64
, aggregation STRING
, optional_limit STRING
)
BEGIN
DECLARE pivotter STRING;
EXECUTE IMMEDIATE (
"SELECT STRING_AGG(' "||aggregation
||"""(IF('||#pivot_col_name||'="'||x.value||'", '||#pivot_col_value||', null)) e_'||`blah.matt.normalize_col_name`(x.value))
FROM UNNEST((
SELECT APPROX_TOP_COUNT("""||pivot_col_name||", #max_columns) FROM `"||table_name||"`)) x"
) INTO pivotter
USING pivot_col_name AS pivot_col_name, pivot_col_value AS pivot_col_value, max_columns AS max_columns;
EXECUTE IMMEDIATE (
'CREATE OR REPLACE TABLE `'||destination_table
||'` AS SELECT '
||(SELECT STRING_AGG(x) FROM UNNEST(row_ids) x)
||', '||pivotter
||' FROM `'||table_name||'` GROUP BY '
|| (SELECT STRING_AGG(''||(i+1)) FROM UNNEST(row_ids) WITH OFFSET i)||' ORDER BY '
|| (SELECT STRING_AGG(''||(i+1)) FROM UNNEST(row_ids) WITH OFFSET i)
||' '||optional_limit
);
END;
So, after above procedures created - below call should now work as expected
CALL `blah.matt.pivot`(
'bigquery-public-data.iowa_liquor_sales.sales' # source table
, 'blah.matt.test2' # destination table
, ['date'] # row_ids
, 'store_number' # pivot_col_name
, 'sale_dollars' # pivot_col_value
, 5 # max_columns
, 'SUM' # aggregation
, '' # optional_limit
);
Note: just simply replace blah and matt with respective proper names of project and dataset
P.S. Obviously you can avoid all this headache and just call Felipe's publicly available proc - fhoffa.x.pivot
But I agree, having those locally will allow to adjust those procs to your potentially specific needs and further improve and code - which I see still quite a room here :o)
This question already has answers here:
Print certain pages only
(2 answers)
Closed 2 years ago.
I can't seem to get Words PrintOut to accept/honor the parameter for PAGES when run in VBScript. Weirdly, it honors COPIES just fine. Any ideas?
Code:
Dim ObjWord
Set ObjWord = CreateObject("Word.Application")
ObjWord.Visible = True
'Open Document
Dim ObjDoc
'https://learn.microsoft.com/en-us/office/vba/api/word.documents.open
'.Open (FileName, ConfirmConversions, ReadOnly, AddToRecentFiles, PasswordDocument, PasswordTemplate, Revert, WritePasswordDocument, WritePasswordTemplate, Format, Encoding, Visible, OpenConflictDocument, OpenAndRepair, DocumentDirection, NoEncodingDialog)
Set ObjDoc = ObjWord.Documents.Open("C:\tmp\test.docx", ,TRUE, , , , , , , , ,TRUE)
'PageRange
'https://learn.microsoft.com/en-us/office/vba/api/word.application.printout
'.PrintOut (Background, Append, Range, OutputFileName, From, To, Item, Copies, Pages, PageType, PrintToFile, Collate, FileName, ActivePrinterMacGX, ManualDuplexPrint, PrintZoomColumn, PrintZoomRow, PrintZoomPaperWidth, PrintZoomPaperHeight)
Dim ObjPrint
ObjPrint = ObjDoc.PrintOut(FALSE, , , , , , , ,"1", , ,TRUE) ' No Error, but Pages not honored
'ObjPrint = ObjDoc.PrintOut(FALSE, , , , , , ,"2", , , ,TRUE) ' Corretly Printes Two Copies
objDoc.Saved = TRUE
objWord.Quit
Set ObjDoc = Nothing
Set objWord = Nothing
So I had to pass an additional param for it to honour the Pages value which was Range.
https://learn.microsoft.com/en-us/office/vba/api/word.document.printout
https://documentation.help/MS-Office-Word-VB/womthPrintOut1.htm
It required a CONSTANT declaration. Eg:
https://learn.microsoft.com/en-us/office/vba/api/word.wdprintoutrange
Specifies a range to print.
WDPRINTOUTRANGE ENUMERATION (WORD)
Name Value Description
wdPrintAllDocument 0 The entire document.
wdPrintCurrentPage 2 The current page.
wdPrintFromTo 3 A specified range.
wdPrintRangeOfPages 4 A specified range of pages.
wdPrintSelection 1 The current selection.
In my case, I believe it will always be 4.
Therefore this worked as expected. (Print Only Page 2)
ObjPrint = ObjDoc.PrintOut(FALSE, ,"4", , , , , ,"2", , ,TRUE)
Further Reading on Printing Word via VBScript. Disclaimer: This is my personal blog --> https://www.freesoftwareservers.com/display/FREES/Print+Microsoft+Word+via+Batch+-+WINWORD.EXE+Switches+-+VBScript+Print+Specific+Pages
In GeoDMS, I use a for_each_nedvc in which the c stands for cdf, however, this does not seem to work.
Does anyone have a suggestion?
container Positief :=
for_each_nedvc(
, OntwikkelPakketten/name
, 'Value(Winst/'+OntwikkelPakketten/name+' / float32(IsPositief/'+OntwikkelPakketten/name+'), Eur)'
, domain
, Eur
, 'Classified_per_OP/'+OntwikkelPakketten/name+'/Target_9K/ClassBreaks'
);
The cdf and url properties cannot be set by a specific for_each operation, but can be set by the generic for_each_ind (with indirect property specification).
For example:
container Positief :=
for_each_ind('nedvc'
, OntwikkelPakketten/name
, 'Value(Winst/'+OntwikkelPakketten/name+' / float32(IsPositief/'+OntwikkelPakketten/name+'), Eur)'
, domain
, Eur
, 'Classified_per_OP/'+OntwikkelPakketten/name+'/Target_9K/ClassBreaks'
);
See also: http://www.objectvision.nl/geodms/operators-a-functions/metascript/for-each-indirect
I have an existing table that resides in one database, and each night I want to completely overwrite all contents of this existing table (TO_TABLE), with all data from another database/table (FROM_TABLE).
Currently I am manually dropping the TO_TABLE and re-writing it using this:
INTO SDE_SPATIAL.GISADMIN.TO_TABLE
FROM GAVIN..AUTH.FROM_TABLE
This works fine, but I would eventually like to convert this into a stored procedure and get this to happen automatically every 24hrs.
Does anyone know how to do the above a bit better and ready for a stored procedure??
UPDATE:
This is the code as it currently stands. I was getting these two errors occur when I run it, the first was"Incorrect Syntax near 'ON'" and the other was "Incorrect Syntax near 'END'".
I removed the ; characters and run again, this time I was getting no errors, and the procedure was being created successfully.
CREATE
PROCEDURE [_ACC_OVERWRITE_PROPERTY_DETAILS]
AS
BEGIN
SET NOCOUNT ON
TRUNCATE TABLE
SDE_SPATIAL.GISADMIN._ACC_TEMP
INSERT
INTO
SDE_SPATIAL.GISADMIN._ACC_TEMP
(
Parcel ,
Assessment ,
House ,
Street ,
St_Type ,
Title ,
Area ,
Area_Units ,
Suburb
)
SELECT
parc.pcl_num ,
parc.ass_num ,
STAD.HOU_NUM ,
stad.str_nme ,
stad.str_typ ,
parc.fmt_ttl ,
aps.property_area ,
LOWER(aps.AREA_INDICATOR) + '²',
stad.sbr_nme
FROM
GAVIN..AUTH.AUPRSTAD stad,
GAVIN..AUTH.AUSRMAST mast,
GAVIN..AUTH.AV_PROPERTY_SUMMARY aps,
GAVIN..AUTH.AUPRPARC parc
WHERE
PARC.PCL_NUM=STAD.PCL_NUM
AND STAD.STR_NUM=MAST.STR_NUM
AND (
PARC.PCL_FLG='R'
OR PARC.PCL_FLG='P')
AND PARC.PCL_NUM=aps.PARCEL_NUMBER
AND stad.SEQ_NUM = 0
END
This is how I would do it with a SQL Server stored procedure (sorry, but I don't think you named your particular SQL dialect):
CREATE PROCEDURE [ResetSpatialData]
AS
BEGIN
SET NOCOUNT ON;
TRUNCATE TABLE SDE_SPATIAL.GISADMIN.TO_TABLE;
INSERT INTO SDE_SPATIAL.GISADMIN.TO_TABLE
([field1], [field2], [field3], [etc], [you get the idea])
SELECT [field1], [field2], [field3], [etc], [you get the idea]
FROM GAVIN..AUTH.FROM_TABLE;
END
You could then schedule this for regular execution on some time basis by using a SQL Server Agent job.
Note: corrected as per #Nick McDermaid's suggestion.
I'm having an issue with Nhibernate version 3.3.3 when trying to use setFirstResult.
IQuery q = session.CreateQuery("Select a from SelectionAssignment a ")
.SetFirstResult(1)
.SetMaxResults(10);
var assignments = q.List<SelectionAssignment>();
The above spits out SQL like this:
SELECT TOP (10) asId30_
, asHostId30_
, asDescript3_30_
, asIsChase30_
, asPosition30_
, asGoalTime30_
, asRoute30_
, asActiveta8_30_
, asPassAssi9_30_
, asSummary10_30_
, asOverrid11_30_
, asDeliver12_30_
, asDirectLoad30_
, asAllowDe14_30_
, asVehicle15_30_
, asCustomerId30_
, asTotalWe17_30_
, asTotalItems30_
, asSingleSKU30_
, asSingleB20_30_
, asCreated21_30_
, asStartDate30_
, asEndDate30_
, asPriority30_
, asIsDeleted30_
, asDeleted26_30_
, asDeleted27_30_ FROM
(
select selectiona0_.Id as Id30_
, selectiona0_.HostId as HostId30_
, selectiona0_.Description as Descript3_30_
, selectiona0_.IsChase as IsChase30_
, selectiona0_.Position as Position30_
, selectiona0_.GoalTime as GoalTime30_
, selectiona0_.Route as Route30_
, selectiona0_.ActivetargetContainer as Activeta8_30_
, selectiona0_.PassAssignment as PassAssi9_30_
, selectiona0_.SummaryPromptType as Summary10_30_
, selectiona0_.OverridePrompt as Overrid11_30_
, selectiona0_.DeliveryLocationId as Deliver12_30_
, selectiona0_.DirectLoad as DirectLoad30_
, selectiona0_.AllowDeliverLocationOverride as AllowDe14_30_
, selectiona0_.VehicleLicense as Vehicle15_30_
, selectiona0_.CustomerId as CustomerId30_
, selectiona0_.TotalWeight as TotalWe17_30_
, selectiona0_.TotalItems as TotalItems30_
, selectiona0_.SingleSKU as SingleSKU30_
, selectiona0_.SingleBatch as SingleB20_30_
, selectiona0_.CreatedDate as Created21_30_
, selectiona0_.StartDate as StartDate30_
, selectiona0_.EndDate as EndDate30_
, selectiona0_.Priority as Priority30_
, selectiona0_.IsDeleted as IsDeleted30_
, selectiona0_.DeletedDate as Deleted26_30_
, selectiona0_.DeletedUser as Deleted27_30_
, ROW_NUMBER() OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
from SelectionAssignment selectiona0_ where ( selectiona0_.IsDeleted=0)
) as query
WHERE query.__hibernate_sort_row > 0
ORDER BY query.__hibernate_sort_row
Whereas if I set the parameter to be .SetFirstResult(0) it works fine.
Can anyone tell me why? Or how I could go about fixing this?
Edit: Apologies, I'm getting an error message of Invalid column name 'asId30' 'asHostId30' etc. The column name is Id, HostId etc.
I discovered the problem. It seemed to be the column name "SelectionAssignment" was causing an issue because NHibernate uses it's own alias on the produced SQL which takes 10 characters and adds a digit. However the 10th and 11th character in my column name are "AS". This seemed to cause a conflict with the query when it tried to put its own AS in there.
Once I changed the column name to be something different "PickingAssignment" the issue was fixed.
I hope I've explained clearly enough and that it can help others with the same issue. Thanks to jbl for trying to help.