I'm trying to automate a process whereby I import multiple files within a hdfs folder. Is there a way like I can get multiple files in one command instead of getting the json and downloading each file with every iteration.
In the below command for the is there a way I can put a wildcard instead of actual file name so I can get multiple files in a single read?
curl -i -L "http://:/webhdfs/v1/?op=OPEN
[&offset=][&length=][&buffersize=]"
I'm using eXist-db over docker container - installing Java over Ubuntu, installing the eXist installation headless jar, and also adding data Volume (Azure file) to store all the physical files and the db data files.
I need to upload automatically files to the eXist-db, after I generate a new file and save it to the volume drive (using C#).
According to the eXist documentation on uploading files there are several methods to upload files to eXist, but none of them work for me.
Dashboard or eXide - not relevant since these are GUI applications.
Java Admin Client - not working because have no GUI -> I'm getting this failure: 'No X11 DISPLAY variable was set, but this program performed an operation which requires it...'
Over REST or WebDAV via web client (using browser or by code), I can run XQuery for queries, but for storing new files, how?
So, the solution I found is to write an XQuery file, using the xmldb:store function.
This query saved the posted file using the specified name and location (in the volume), and the stored file can then be retrieved via REST or WebDAV.
But I feel that there must be a simpler solution...
Can anyone help?
BTW, here is the xmldb:store XQuery:
xquery version "3.1";
declare function local:upload() {
let $filename := request:get-uploaded-file-name("file")
let $log-in := xmldb:login("/db", "Admin", "admin")
let $file := "file:///usr/new_file_location.xml"
let $record := doc($file)
let $store := xmldb:store("/db/akn", "new_file_name.xml", $record)
return
<results>
<message>File {$file} has been stored.</message>
</results>
};
local:upload()
When starting eXist as described in the eXist Docker documentation - with it listening on port 8080 - you can access all of eXist's standard endpoints:
http://localhost:8080/exist/webdav/db for WebDAV
http://localhost:8080/exist/rest/db for REST
http://localhost:8080/exist/xmlrpc/db for XML-RPC
http://localhost:8080/exist/apps for apps installed in /db/apps.
Of course if you've configured Docker's eXist to listen on a different port, just switch the port.
Thus, to upload files to a Dockerized eXist programmatically, the methods outlined in the documentation article you referenced, Uploading files, should all work: WebDAV, client.sh, Ant, or even curl. For WebDAV, if you haven't configured users and passwords, you'd just connect with the URL http://localhost:8080/exist/webdav/db, username "admin", and a blank password. For Ant, see the Ant tasks documentation. For curl, you would perform an HTTP PUT request to the REST interface:
curl -s -f -H 'Content-Type: application/xml' \
-T <filename> \
--user <user>:<password> \
"http://localhost:8080/exist/rest/db/apps/<collection>/<filename>"
This is also possible:
echo put /home/files/<FILRPATH>/main.xml | /usr/local/eXist-db/
bin/client.sh -s
Is there any way to store or update a CSV file to One-drive after Jenkins build run? I want it to happen without any third-party plugin.
You can do this using OneDrive API and curl, e.g.:
curl https://graph.microsoft.com/v1.0/me/drive/root:/data.csv:/content -X PUT -d #data.csv -H "Authorization: bearer access_token_here"
One limitation is that your csv file couldn't be more than 4MB.
When adding a URL into a web browser, I get the usual prompt to open the XML file and view it. However, when I use the same URL within a Curl batch file it only appears to download the login aspx page.
//stuff/stuff/Report.aspx?Report=All_Nodes_IP_Report&DataFormat=XML&AccountID=<UID>&Password=<password>
My batch file looks like this:
curl -L "//stuff/stuff/Report.aspx?Report=All_Nodes_IP_Report&DataFormat=XML&AccountID=<UID>&Password=<Password>" -o "local.xml" -v
pause
What am I doing wrong? There's no proxy server between me and the report URL..? The web site is https but I can't include that as the validation checker keeps moaning at me :)
why use CURL when you can use one application called MGET that i create.
Download Link:
http://bit.ly/1i1FpGE
Syntax of the command:
MGET //stuff/stuff/Report.aspx?Report=All_Nodes_IP_Report&DataFormat=XML&AccountID=<UID>&Password=<Password> local.xml
And if you want to use HTTPS do it, for best experience use HTTP
I'm a new bee to the rails api and curl commands.
I have a rails api which response to request sends from the native app. And to one of the requests sent from native api using curl command my rails api is sending file saved to the database to that desktop app.
But I'm not getting how to save that file using curl commands when I use curl -o/-O commands only file object is saved (example: #) and not the actual file. When I call the same rails method on the browser I get the file downloaded properly.
Thanks in advance
UPDATE
I can save those file types which can be displayed inline on the browser but not all.
You can use any shell command and save it into the file by > file_name command. In your case it will be
curl http://example.com > filename