I am trying to move data from InfluxDB to QuestDB,
I was able to export my tables as JSON by following: https://stackoverflow.com/a/27913640/1267728
How do I now import these JSON files into QuestDB?
Convert from JSON to CSV
QuestDB supports importing data via CSV file, so first you would need to flatten the JSON and ensure that column names are modified to reflect nested properties.
There is a Java library called Json2Flat that already does this.
Import the CSV file
Using the REST API, you can import the data into QuestDB
curl -F data=file.csv http://localhost:9000/imp
For more details of how to use the REST API, please go to the official documentation.
Check the data
To verify that the import is successful, you can check via the Web Console or via CURL…
curl -G --data-urlencode "query=select * from 'file.csv'" http://localhost:9000/exp
Just adding here that QuestDB recently improved the performance of CSV ingestion. More info at https://questdb.io/docs/guides/importing-data/
If you want to avoid converting from JSON (and probably more performant as well than exporting to JSON for large tables), you can use the influxd inspect export-lp command that exports all your data as ILP points. You can choose to export a single bucket.
Once you have the ILP files, you can import as explained at this other StackOverflow post What's the best way to upload an ILP file into QuestDB?
Related
As we all know,joern can easily analysis project written by c/c++,and it will generate CPG(code property graph).
Now I hava 2 csv file(nodes.csv and rels.csv),the 2 csv file can build AST(abstract syntax code)by neo4j and it can be shown in neo4j.But AST is not enough for me,what i need is CPG.How could i use the 2 csv file to build a CPG and show it in neo4j.I wanna to know how to use joern to make my ast'csv file become cpg'csv file.Or you hava other ways to make me do this.
I relly need your help,please.Thank you very much.
Hello i am trying to import csv file using neo4j-admin tool but i am receiving this error graph.db' already contains a database
Does this mean that i need to create new database every new import ?
neo4j-admin import is only for creating a new db, it cannot be used to add to an existing db, so you would need to delete your graph.db folder prior to import.
You'll need to use an alternate means, like a LOAD CSV query if you want to add into an existing graph.
I am currently writing some Java code extracting some data and writing them as Linked Data, using the TRIG syntax. I am now using Jena, and Fuseki to create a SPARQL endpoint to query and visualize this data.
The data is written so that each source dataset gives me a .trig file, containing one named graph. So I want to load thoses files in Fuseki. Except that it doesn't seem to understand the Trig syntax...
If I remove the named graphs, and rename the files as .ttl, everything loads perfectly in the default graphs. But if I try to import trig files :
using Fuseki's webapp uploader, it either crashes ("Can't make new graphs") or adds nothing except the prefixes, as if the graphs other than the default ones could not be added (the logs say nothing helpful except the error code and description).
using Java code, the process is too slow. I used this technique : " Loading a .trig file into TDB? " but my trig files are pretty big, so this solution is not very good for me.
So I tried to use the bulk loader, the console command 'tdbloader'. This time everything seems fine, but in the webapp, there is still no data.
You can see the process going fine here : Quads are added just fine
But the result still keeps only the default graph and its original data : Nothing is added
So, I don't know what to do. The guys behind Jena and Fuseki suggested not to use the bulk loader in the Java code (rather than the command line tool), so that's one solution I guess I'd like to avoid.
Did I miss something obvious about how to load TRIG files to Fuseki? Thanks.
UPDATE :
As it seemed to be a problem in my configuration (see the comments of this post for a link to my config file; I cannot post more than 2 links), I tried to add some kind of specification for some named graphs I would like to see added to the dataset on Fuseki.
I added code to link (with ja:namedgraph) external graphs that I added via tdbloader. This seems to work. Great!
Now another problem : there's no inference, even when my config file specifies an Inference model... I set that queries should be applied with named graphs merged as the default graph, but this does not seem to carry the OWL Inference rules...So simple queries work, but I have 1/ to specify the graph I query (with "FROM") and 2/ no inference in my data.
The two methods are to use the tdb bulkloader offline or you can POST data into the dataset directly. (i.e. HTTP POST operations to http://localhost:3030/ds).
You can test where your graph are there with a query like
SELECT (count(*) AS ?C) { GRAPH ?g { ?s ?p ?o } }
The named graphs will show up when the Fuseki server is started unless your configuration of the SPARQL services only exports one graph.
I have imported the file in mallet, now I want to model topic from the imported data and store them in a text file, from where I will be able to read those topics. Can anyone help in writing the commands for topic extraction, as I typed command below for topic extraction but it throws exception.
bin\mallet import-dir --input D:\Data\test1 --output test1.mallet --keep-sequence --remove-stopwords --extra-stopwords extra.txt
by removing --keep-sequence --remove-stopwords --extra-stopwords extra.txt i am able to import file after that, when I try to train model exception is thrown.
I recommend you to use GUI for mallet.
https://code.google.com/p/topic-modeling-tool/
So it looks like the gremlin API requires a url to import a GraphML file to the server (http://docs.neo4j.org/chunked/stable/gremlin-plugin.html#rest-api-load-a-sample-graph). I was hoping there'd be some API where you could just POST the GraphML to it, does something like this exist?
I realise I could write a Neo4j extension to essentially do this, but I was wondering if one already existed...
There a shell extension at https://github.com/jexp/neo4j-shell-tools#graphml-import providing this feature. It should not be too hard to convert that into a server extension.
If the graph is not huge, perhaps you can try passing the file as a string to the gremlin extension and use the script in the doc you cited. Therefore your gremlin script expects a String variable that contains your graph and it creates the file (by writing the string graph to the file):
def fos= new FileOutputStream('path_to_my_file.xml')
fos.write(myGraphAsString)
You can then load this file:
g.clear()
g.loadGraphML('file:/path_to_my_file.xml')