I am new in Spring 5 and Reactive Programming. My problem is creating the export feature for the database by a rest API.
User hits GET request -> Server reads data and returns data as a zip file. Because zip file is large, so I need to stream these data.
My code as below:
#GetMapping(
value = "/export",
produces = ["application/octet-stream"],
headers = [
"Content-Disposition: attachment; filename=\"result.zip\"",
"Content-Type: application/zip"])
fun streamData(): Flux<Resource> = service.export()
I use curl as below:
curl http://localhost/export -H "Accept: application/octet-stream"
But it always returns 406 Not Acceptable.
Anyone helps?
Thank you so much
The headers attribute of the #GetMapping annotation are not headers that should be written to the HTTP response, but mapping headers. This means that your #GetMapping annotation requires the HTTP request to contain the headers you've listed. This is why the request is actually not mapped to your controller handler.
Now your handler return type does not look right - Flux<Resource> means that you intend to return 0..* Resource instances and that they should be serialized. In this case, a return type like ResponseEntity<Resource> is probably a better choice since you'll be able to set response headers on the ResponseEntity and set its body with a Resource.
Is it right, man? I still feel it's not good with this solution at the last line when using blockLast.
#GetMapping("/vehicle/gpsevent", produces = ["application/octet-stream"])
fun streamToZip(): ResponseEntity<FileSystemResource> {
val zipFile = FileSystemResource("result.zip")
val out = ZipOutputStream(FileOutputStream(zipFile.file))
return ResponseEntity
.ok().cacheControl(CacheControl.noCache())
.header("Content-Type", "application/octet-stream")
.header("Content-Disposition", "attachment; filename=result.zip")
.body(ieService.export()
.doOnNext { print(it.key.vehicleId) }
.doOnNext { it -> out.putNextEntry(ZipEntry(it.key.vehicleId.toString() + ".json")) }
.doOnNext { out.write(it.toJsonString().toByteArray(charset("UTF-8"))) }
.doOnNext { out.flush() }
.doOnNext { out.closeEntry() }
.map { zipFile }
.doOnComplete { out.close() }
.log()
.blockLast()
)
}
Related
What I want to do is really what the title says. I would like to know how I can receive data per post in hyper, for example, suppose I execute the following command (with a server in hyper running on port :8000):
curl -X POST -F "field=#/path/to/file.txt" -F "tool=curl" -F "other-file=#/path/to/other.jpg" http://localhost:8000
Now, I'm going to take parf of the code on the main page of hyper as an example:
use std::{convert::Infallible, net::SocketAddr};
use hyper::{Body, Request, Response, Server};
use hyper::service::{make_service_fn, service_fn};
async fn handle(_: Request<Body>) -> Result<Response<Body>, Infallible> {
Ok(Response::new("Hello, World!".into()))
}
#[tokio::main]
async fn main() {
let addr = SocketAddr::from(([127, 0, 0, 1], 8000));
let make_svc = make_service_fn(|_conn| async {
Ok::<_, Infallible>(service_fn(handle))
});
let server = Server::bind(&addr).serve(make_svc);
if let Err(e) = server.await {
eprintln!("server error: {}", e);
}
}
So, now, with this basic code, how can I receive the data per post that my curl command above would send? How do I adapt my code to read the data? I've tried to search the internet, but what I found was that hyper doesn't actually split the request body depending on the HTTP method, it's all part of the same body. But I haven't been able to find a way to process data like the above with code like mine. Thanks in advance.
Edit
I tried the exact code that they left me in the answer. That is, this code:
async fn handle(req: Request<Body>) -> Result<Response<Body>, Infallible> {
let mut files = multipart::server::Multipart::from(req);
.....
}
But I get this error:
expected struct multipart::server::Multipart, found struct
hyper::Request
How can I solve that?
It is a single body, but the data is encoded in a way that contains the multiple files.
This is called multipart, and in order to parse the body correctly you need a multipart library such as https://crates.io/crates/multipart
To hyper integration you need to add the feature flag hyper in Cargo.toml
multipart = { version = "*", features = ["hyper"] }
Then
async fn handle(mut files: multipart::server::Multipart) -> Result<Response<Body>, Infallible> {
files.foreach_entry(|field| {
// contains name, filename, type ..
println!("Info: {:?}",field.headers);
// contains data
let mut bytes:Vec<u8> = Vec::new();
field.data.read_to_end(&mut bytes);
});
Ok(Response::new("Received the files!".into()))
}
You can also use it like this
async fn handle(req: Request<Body>) -> Result<Response<Body>, Infallible> {
let mut files = multipart::server::Multipart::from(req);
.....
}
I want to upload a txt file to a website using a POST request with HTTPBuilder and multipart/form-data
I've tried running my function and I get a HTTP 200 OK response, but the file doesn't appear on the website anywhere.
private Map fileUpload(String url, File file){
log.debug "doPost: $url body: ${file.getName()}"
FileBody fileBody = new FileBody(file,ContentType.APPLICATION_OCTET_STREAM)
def result = [:]
try {
def authSite = new HTTPBuilder(url)
authSite.auth.basic(user, password)
authSite.request(POST) { req ->
headers.Accept = "application/json, text/javascript, */*; q=0.01"
req.params.setParameter(CoreConnectionPNames.SO_TIMEOUT, 20000)
req.params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 60000)
def mpe = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
mpe.addPart("gxt",fileBody)
req.setEntity(mpe)
response.success = { resp, reader ->
result = reader
}
response.failure = { resp, reader ->
println "My response handler got response: ${resp.statusLine}"
}
}
}
catch (e) {
log.debug("Could not perform POST request on URL $url", e)
throw e
}
return result
}
From debugging this is the status recieved
3695 [main] DEBUG org.apache.http.wire - << "HTTP/1.1 200 OK[\r][\n]"
3695 [main] DEBUG org.apache.http.wire - << "Date: Thu, 10 Jan 2019 07:34:06 GMT[\r][\n]"
Anything I'm doing wrong? I don't get any errors but it just seems like nothing happens.
I don't have anything conclusive, but I suspect there is something invalid with the way you set up the multipart upload.
To help figure this out, below is a standalone, working, multipart upload groovy script using HttpBuilder:
#Grab('org.codehaus.groovy.modules.http-builder:http-builder:0.7.1')
#Grab('org.apache.httpcomponents:httpmime:4.2.1')
import org.apache.http.entity.mime.content.*
import org.apache.http.entity.mime.*
import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.Method.POST
fileUpload('https://httpbin.org/post', new File('data.txt'))
Map fileUpload(String url, File file){
println "doPost: $url body: ${file.name}"
def result
try {
new HTTPBuilder(url).request(POST) { req ->
requestContentType = "multipart/form-data"
def content = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
content.addPart(file.name, new InputStreamBody(file.newInputStream(), file.name))
req.entity = content
// json might be something else (like a reader)
// depending on the response content type
response.success = { resp, json ->
result = json
println "RESP: ${resp.statusLine}, RESULT: $json"
}
response.failure = { resp, json ->
println "My response handler got response: ${resp.statusLine}"
}
}
} catch (e) {
println "Could not perform POST request on URL $url"
throw e
}
result
}
The script assumes a file data.txt with the data to post in the current directory. The script posts to httpbin.org as a working test endpoint, adjust accordingly to post to your endpoint instead.
Saving the above in test.groovy and executing will yield something like:
~> groovy test.groovy
doPost: https://httpbin.org/post body: data.txt
RESP: HTTP/1.1 200 OK, RESULT: [args:[:], data:, files:[data.txt:{ "foo": "bar" }], form:[:], headers:[Accept:*/*, Connection:close, Content-Type:multipart/form-data; boundary=ZVZuV5HAdPOt2Sv7ZjxuUHjd8sDAzCz9VkTqpJYP, Host:httpbin.org, Transfer-Encoding:chunked], json:null, origin:80.252.172.140, url:https://httpbin.org/post]
(note that first run will take a while as groovy grapes need to download the http-builder dependency tree)
perhaps starting with this working example and working your way back to your code would help you pinpoint whatever is not working in your code.
For outgoing HTTP requests (using meteor.http.call) I can define params and/or data. The results are then available (via results.content).
How do I access and parse the body/contents/data of incoming HTTP requests?
Using iron-router, I have got as far as this:
Router.map(function () {
this.route('httpTest', {
path: '/httpTest',
action: function () {
var request = this.request;
var response = this.response;
console.log('request_body: '+ request.body);
// request.body does not work. what should it be????
N.B. I understand that I CAN access query parameters, but I want to access form data and/or json data from the body of an incoming http request.
The request is an incoming http message, which is a Readable Stream, so you can get the data of the request by reading from that stream.
The following should work (but I haven't tested it):
var readable = this.request;
var alldata = "";
readable.on('data', function(chunk) {
alldata += chunk;
})
readable.on('end', function() {
console.log('do something with alldata');
});
It may not be working because of the missing where: 'server'. Here is a working example:
Router.map(function() {
this.route('test', {
where: 'server',
action: function() {
console.log(this.request.body.make);
console.log(this.request.body.model);
this.response.writeHead(200, {'Content-Type': 'text/html'});
this.response.end('hello!\n');
}
});
});
From the command line I can hit this route with:
curl -X POST -H "Content-Type: application/json" -d '{"make":"honda","model":"civic"}' http://localhost:3000/test
Which prints the expected honda and civic in the server's terminal. It looks like this.request.body is already parsed so you can access any variables directly which is nice if your input is json.
To read the raw body, without having Node.js JSON-ify it, in a synchronous way, I used this:
Router.route("my/api", function() {
var rawBody = "";
var chunk;
while ((chunk = this.request.read()) !== null) {
rawBody += chunk;
}
}, { where: "server" });
(the asynch way proposed in another answer here didn't worked for me, although it should as per Node.js documentation).
I'm trying to delete an entry from the database by odata. I get the error message
{"error":{"code":"","message":{"lang":"en-US","value":"Bad Request - Error in query syntax."}}}
my code:
function deleteMonthEntry() {
var item = actMonthEntries.getItem(listIndex);
var queryString = "Stundens(" + item.data.datensatz_id + ")?$format=json";
var requestUrl = serviceUrl + queryString;
WinJS.xhr({
type: "delete",
url: requestUrl,
headers: {
"Content-type": "application/json"
}
}).done(
function complete(response) {
},
function (error) {
console.log(error);
}
);
}
My request URL looks like this:
requestUrl = "http://localhost:51893/TimeSheetWebservice.svc/Stundens(305233)?$format=json"
Thanks
Marlowe
At least I found the solution:
I've entered an filter request to my service like this:
TimeSheetWebservice.svc/Stundens?$filter=datensatz_id eq 305221
this returned the correct entry with this link:
TimeSheetWebservice.svc/Stundens(305221M)
So if I enter a M after the ID, everything works fin. But I have no idea where this M comes from.
Can anyone tell me the reason for this M? It does not belong to the ID. The ID is this
305221
Marlowe
Are you sure the server you're talking to supports the $format query option? Many don't. I would try removing that part of the request URI, and instead modify your headers value to specify an Accept header:
headers: {
"Content-Type": "application/json",
"Accept": "application/json"
}
For servers where $format is allowed, giving it a json value is equivalent to providing an Accept header with the application/json MIME type.
In general, for a DELETE operation, the Accept header or $format value only matters for error cases. With a successful DELETE, the response payload body will be empty, so there's no need for the server to know about your format preference.
I'm currently developing a Firefox add-on(using https://addons.mozilla.org/en-US/developers/docs/sdk/1.0/ ) that consumes an API where the return data is in xml.
My problem is that I need to parse the returned data, and would like to do that using a xml object.
Since the request module only supports JSON and Text ( https://addons.mozilla.org/en-US/developers/docs/sdk/1.0/packages/addon-kit/docs/request.html#Response ) I need to convert the response.text to XML.
The code looks like this:
var Request = require('request').Request
.......
var req = Request({
url: https://to-the-api.com,
content: {
op: 'get-the-data-op',
password: "super-sec",
user: "username"
},
onComplete: function (response) {
dataAsText = response.text;
console.log("output: " + dataAsText);
}
});
req.post();
I have tried to user (new DOMParser).parseFromString(response.text, 'text/xml') but unfortunately it just fails with a error like ReferenceError: DOMParser is not defined
The question is if anyone of you guys have been able to create a Xml object inside a Firefox add-on, and if so, how?
Looks like the capability to parse response as xml was present, but has been removed. check out this bugzilla reference
Can't you use a normal XMLHttpRequest if you want to process the response as XML?
If DOMParser is unavailable you can try E4X:
var xml = new XML(response.text);
alert(xml.children().length());
You want to use the XMLHttpRequest object to handle your xhr request. Then when you get a response back access the responseXML object of the request variable. In the responseXML you'll have the documentElement and can use the querySelectorAll or querySelector to find elements you want. In each element you want just grab the textContent you need.
Here's an example to get you going (this looks for the 'xmls' element in the response):
var request = new require("xhr").XMLHttpRequest();
request.open('GET', 'https://to-the-api.com', true);
request.onreadystatechange = function (aEvt) {
if (request.readyState == 4) {
if(request.status == 200) {
var xmls = request.responseXML.documentElement.querySelectorAll("xmls");
for (var i = 0; i < xmls.length; i++) {
console.log("xml", i, xmls[i], xmls[i].textContent);
}
}
else {
console.log('Error', request.responseText);
}
}
};
request.send(null);