Uploading txt file via POST request with HttpBuilder - post

I want to upload a txt file to a website using a POST request with HTTPBuilder and multipart/form-data
I've tried running my function and I get a HTTP 200 OK response, but the file doesn't appear on the website anywhere.
private Map fileUpload(String url, File file){
log.debug "doPost: $url body: ${file.getName()}"
FileBody fileBody = new FileBody(file,ContentType.APPLICATION_OCTET_STREAM)
def result = [:]
try {
def authSite = new HTTPBuilder(url)
authSite.auth.basic(user, password)
authSite.request(POST) { req ->
headers.Accept = "application/json, text/javascript, */*; q=0.01"
req.params.setParameter(CoreConnectionPNames.SO_TIMEOUT, 20000)
req.params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 60000)
def mpe = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
mpe.addPart("gxt",fileBody)
req.setEntity(mpe)
response.success = { resp, reader ->
result = reader
}
response.failure = { resp, reader ->
println "My response handler got response: ${resp.statusLine}"
}
}
}
catch (e) {
log.debug("Could not perform POST request on URL $url", e)
throw e
}
return result
}
From debugging this is the status recieved
3695 [main] DEBUG org.apache.http.wire - << "HTTP/1.1 200 OK[\r][\n]"
3695 [main] DEBUG org.apache.http.wire - << "Date: Thu, 10 Jan 2019 07:34:06 GMT[\r][\n]"
Anything I'm doing wrong? I don't get any errors but it just seems like nothing happens.

I don't have anything conclusive, but I suspect there is something invalid with the way you set up the multipart upload.
To help figure this out, below is a standalone, working, multipart upload groovy script using HttpBuilder:
#Grab('org.codehaus.groovy.modules.http-builder:http-builder:0.7.1')
#Grab('org.apache.httpcomponents:httpmime:4.2.1')
import org.apache.http.entity.mime.content.*
import org.apache.http.entity.mime.*
import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.Method.POST
fileUpload('https://httpbin.org/post', new File('data.txt'))
Map fileUpload(String url, File file){
println "doPost: $url body: ${file.name}"
def result
try {
new HTTPBuilder(url).request(POST) { req ->
requestContentType = "multipart/form-data"
def content = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
content.addPart(file.name, new InputStreamBody(file.newInputStream(), file.name))
req.entity = content
// json might be something else (like a reader)
// depending on the response content type
response.success = { resp, json ->
result = json
println "RESP: ${resp.statusLine}, RESULT: $json"
}
response.failure = { resp, json ->
println "My response handler got response: ${resp.statusLine}"
}
}
} catch (e) {
println "Could not perform POST request on URL $url"
throw e
}
result
}
The script assumes a file data.txt with the data to post in the current directory. The script posts to httpbin.org as a working test endpoint, adjust accordingly to post to your endpoint instead.
Saving the above in test.groovy and executing will yield something like:
~> groovy test.groovy
doPost: https://httpbin.org/post body: data.txt
RESP: HTTP/1.1 200 OK, RESULT: [args:[:], data:, files:[data.txt:{ "foo": "bar" }], form:[:], headers:[Accept:*/*, Connection:close, Content-Type:multipart/form-data; boundary=ZVZuV5HAdPOt2Sv7ZjxuUHjd8sDAzCz9VkTqpJYP, Host:httpbin.org, Transfer-Encoding:chunked], json:null, origin:80.252.172.140, url:https://httpbin.org/post]
(note that first run will take a while as groovy grapes need to download the http-builder dependency tree)
perhaps starting with this working example and working your way back to your code would help you pinpoint whatever is not working in your code.

Related

Kinesis Firehose HTTP_Endpoint destination Response format

What is the right format of the Response for Kinesis Firehose with http_endpoint as destination. Have already gone through the aws link:
https://docs.aws.amazon.com/firehose/latest/dev/httpdeliveryrequestresponse.html#responseformat
I have used the below lambda code in python(integrated in api) as well as with many other options, but keep getting the below error message. The test is performed using the "Test with Demo Data" option
sample code:
def lambda_handler(event, context):
data ={}
headersD = {}
headersD['content-length'] = 0
headersD['content-type'] = 'application/json'
data['requestId'] = 'ed4acda5-034f-9f42-bba1-f29aea6d7d8f'
data['timestamp'] = '1578090903599'
bodyDetail= {}
data['body'] = ''
data['headers'] =headersD
data['statusCode']=200
resp = json.dumps(data)
return resp
error response as seen in the logs:
The response received from the endpoint is invalid. See Troubleshooting HTTP Endpoints in the Firehose documentation for more information. Reason:. Response for request 'request-Id' is not recognized as valid JSON or has unexpected fields. Raw response received: 200 "HttpEndpoint.InvalidResponseFromDestination"
Here is the sample output that worked(in python):
responseBody = {
"requestId": "requestId",
"timestamp": 123456
}
resp = {
"headers": {"Content-Type": "application/json", "Content-Length": 100},
"body": json.dumps(responseBody),
"statusCode": 200
}
return resp

Downloading file from POST request with Elm

I'm in the position that I have an HTTP POST endpoint /render that returns a PDF document, and would like to present a button/link to the user that will cause this document to be downloaded and saved to a file without navigating away from my Elm app.
Ideally the POST will accept a text/plain body with a custom format, but I could rework the endpoint to accept multipart/form-data or application/x-www-form-urlencoded.
I can download the raw data to the Elm app successfully as follows, but I'm at a loss for how to save the file to disk.
import Http
render : String -> Http.Request String
render body =
Http.request
{ method = "POST"
, headers = []
, url = "/render"
, body = Http.stringBody "text/plain" body
, expect = expectString
, timeout = Nothing
, withCredentials = False
}
I did it using expectBytes rather then expectString
so my code is
import Bytes exposing (Bytes)
import File.Download as Download
Msg = .. | FormUploaded (Result Http.Error Bytes)
Http.post
{ url = "/passports"
, body =
Http.multipartBody...
, expect = Http.expectBytesResponse FormUploaded (resolve Ok)
}
downloadPdf : Bytes -> Cmd msg
downloadPdf pdfContent =
Download.bytes "form.pdf" "application/pdf" pdfContent
update : Msg -> Model -> ( Model, Cmd Msg )
update model =
...
FormUploaded (Ok response) ->
( model, downloadPdf response )
FormUploaded (Err err) ->
( model, Cmd.none )
-- this helper function copied from https://github.com/elm/http/blob/2.0.0/src/Http.elm#L514-L521
resolve : (body -> Result String a) -> Http.Response body -> Result Http.Error a
resolve toResult response =
case response of
BadUrl_ url ->
Err (BadUrl url)
Timeout_ ->
Err Timeout
NetworkError_ ->
Err NetworkError
BadStatus_ metadata _ ->
Err (BadStatus metadata.statusCode)
GoodStatus_ _ body ->
Result.mapError BadBody (toResult body)
it's not ideal but works
PS: I got help from Elm Slack Channel https://elmlang.slack.com/

Response zip file with WebFlux

I am new in Spring 5 and Reactive Programming. My problem is creating the export feature for the database by a rest API.
User hits GET request -> Server reads data and returns data as a zip file. Because zip file is large, so I need to stream these data.
My code as below:
#GetMapping(
value = "/export",
produces = ["application/octet-stream"],
headers = [
"Content-Disposition: attachment; filename=\"result.zip\"",
"Content-Type: application/zip"])
fun streamData(): Flux<Resource> = service.export()
I use curl as below:
curl http://localhost/export -H "Accept: application/octet-stream"
But it always returns 406 Not Acceptable.
Anyone helps?
Thank you so much
The headers attribute of the #GetMapping annotation are not headers that should be written to the HTTP response, but mapping headers. This means that your #GetMapping annotation requires the HTTP request to contain the headers you've listed. This is why the request is actually not mapped to your controller handler.
Now your handler return type does not look right - Flux<Resource> means that you intend to return 0..* Resource instances and that they should be serialized. In this case, a return type like ResponseEntity<Resource> is probably a better choice since you'll be able to set response headers on the ResponseEntity and set its body with a Resource.
Is it right, man? I still feel it's not good with this solution at the last line when using blockLast.
#GetMapping("/vehicle/gpsevent", produces = ["application/octet-stream"])
fun streamToZip(): ResponseEntity<FileSystemResource> {
val zipFile = FileSystemResource("result.zip")
val out = ZipOutputStream(FileOutputStream(zipFile.file))
return ResponseEntity
.ok().cacheControl(CacheControl.noCache())
.header("Content-Type", "application/octet-stream")
.header("Content-Disposition", "attachment; filename=result.zip")
.body(ieService.export()
.doOnNext { print(it.key.vehicleId) }
.doOnNext { it -> out.putNextEntry(ZipEntry(it.key.vehicleId.toString() + ".json")) }
.doOnNext { out.write(it.toJsonString().toByteArray(charset("UTF-8"))) }
.doOnNext { out.flush() }
.doOnNext { out.closeEntry() }
.map { zipFile }
.doOnComplete { out.close() }
.log()
.blockLast()
)
}

Groovy httpbuilder post list params

I'm trying to consume a web service from my grails project. I'm using httpbuilder 0.7.2. Below is my http client.
static def webServiceRequest(String baseUrl, String path, def data,method=Method.GET,contentType=ContentType.JSON){
def ret = null
def http = new HTTPBuilder(baseUrl)
http.request(method, contentType) {
uri.path = path
requestContentType = ContentType.URLENC
if(method==Method.GET)
uri.query = data
else
body = data
headers.'User-Agent' = 'Mozilla/5.0 Ubuntu/8.10 Firefox/3.0.4'
response.success = { resp, json ->
println "response status: ${resp.statusLine}"
ret = json
println '--------------------'
}
}
return ret
}
The issue is coming when i'm trying to send something like this:
def input = [:]
input['indexArray'] = [1,5]
api call
def response = webServiceRequest(url,uri,input,Method.POST)
when i'm printing the value of post data in my server it shows only last value of list.
{"indexArray":"5"}
it should show both 1 and 5
If you want to send json data using contenttype application/x-www-form-urlencoded you have to explicitly convert the data before adding it to the body, you can use (data as JSON).
I am using RESTClient (nice convenience wrapper on HTTPBuilder, https://github.com/jgritman/httpbuilder/wiki/RESTClient). It is as simple as this with Spock.
RESTClient restClient = new RESTClient("http://localhost:8080")
restClient.contentType = ContentType.JSON
Also it automatically parses the JSON data, so my Spock test is:
when: "we check the server health"
HttpResponseDecorator response = restClient.get([path : "/health"]) as HttpResponseDecorator
then: "it should be up"
response != null
200 == response.status
'application/json' == response.contentType

Groovy Httpbuilder authentication with Realm + RABBIT MQ

I am trying to to make a rabbitmq http api call to know how queues are there and other infos...
I need 3 variables to pass on to the api
1) url: (http://localhost:55672/api) 2) username/password: guest/guest
3) realm: "RabbitMQ Management" //i am not sure if this is important
4) path: "/queues"
when i make curl statement it gives a positive response
sudo curl -i -u guest:guest (http://localhost:55672)/api/queues
HTTP/1.1 200 OK
Server: MochiWeb/1.1 WebMachine/1.7 (participate in the frantic)
Date: Tue, 03 Jul 2012 01:39:05 GMT
Content-Type: application/json
Content-Length: 6176
Cache-Control: no-cache
but using httpbuilder from groovy. here is the code
def http = new HTTPBuilder("(http://localhost:55672/api)")
http.auth.basic 'guest','guest'
http.request(GET) { req ->
uri.path = '/queues'
response.success = { resp, reader ->
assert resp.statusLine.statusCode == 200
println "Got response: ${resp.statusLine}"
println "Content-Type: ${resp.headers.'Content-Type'}"
println reader.json
}
response.'404' = { println 'Not found' }
}
I am getting "not found" as the result. I am not including realm because I am unable to if i can insert "realm" in httpbuilder. it only comes with OAuth however I need to use basic auth for rabbit mq http api calls.
Does anyone knows how to include realm name in httpbuilder groovy for basic authentication? is there any other way. Kindly let me know! thanks!
Does this work?
def http = new HTTPBuilder( 'http://localhost:55672' )
http.auth.basic 'guest','guest'
http.request(GET) { req ->
uri.path = '/api/queues'
response.success = { resp, reader ->
assert resp.statusLine.statusCode == 200
println "Got response: ${resp.statusLine}"
println "Content-Type: ${resp.headers.'Content-Type'}"
println reader.json
}
response.'404' = { println 'Not found' }
}
Took the braces and the path out of your base url, added /api to the path

Resources