I'm trying to test my Grails web application by creating and sending a multipart request from a stand-alone groovy test script that's built by gradle. But I'm struggling.
I can't attach a custom Content-ID header
I can't attach a file of random bytes created at runtime (I can attach an existing file, but I need many random files of varying size)
EDIT (Thanks to Xeon):
My script is now sending a valid multipart request, but my grails web app is not accepting any headers other than "Content-Type" for some reason.
Heres my code:
The Stand-Alone Test Script code:
void sendMultipartRequest(String url) {
HTTPBuilder httpBuilder = new HTTPBuilder(url)
httpBuilder.request(Method.POST){ req ->
MultipartEntityBuilder entityBuilder = new MultipartEntityBuilder()
entityBuilder.setBoundary("----boundary")
entityBuilder.setMode(HttpMultipartMode.RFC6532)
String randomString = myGenerateRandomStringMethod()
FormBodyPart formBodyPart = new FormBodyPart(
"SOME_NAME",
new InputStreamBody(new ByteArrayInputStream(randomString.bytes), "attachment", "SOME_NAME")
)
formBodyPart.addField("Content-ID", "abc123")
entityBuilder.addPart(formBodyPart)
response.success = { resp ->
println("Success with response ${resp.toString()}")
}
response.failure = { resp ->
println("Failure with response ${resp.toString()}")
}
delegate.setHeaders(["Content-Type":"multipart/related; boundary=----boundary"])
req.setEntity(entityBuilder.build())
}
}
Grails web-app side in the controller for handling posts:
def submitFiles() {
if(request instanceof MultipartHttpServletRequest){
HashMap<String, Byte[]> fileMap = extractMultipartFiles(request)
someService.doStuffWith(fileMap)
}
}
private HashMap<String, Byte[]> extractMultipartFiles(MultipartHttpServletRequest multipartRequest) {
HashMap<String, Byte[]> files = new HashMap<>()
for(element in mulipartRequest.multiFileMap){
MultipartFile file = element.value.first()
String contentId = multipartRequest.getMultipartHeaders(element.key).get("Content-ID")?.first()
if(contentId) files.put(contentId, file.getBytes())
}
return files
}
Libraries I'm using:
ext {
groovyVersion = "2.3.4"
commonsLangVersion = "2.6"
httpBuilderVersion = "0.7.1"
httpmimeVersion = "4.3.4"
junitVersion = "4.11"
}
dependencies {
compile "org.codehaus.groovy:groovy-all:${groovyVersion}"
compile "commons-lang:commons-lang:${commonsLangVersion}"
compile "org.codehaus.groovy.modules.http-builder:http-builder:${httpBuilderVersion}"
compile "org.apache.httpcomponents:httpmime:${httpmimeVersion}"
testCompile group: 'junit', name: 'junit', version: "${junitVersion}"
}
You can always use some subclass of ContentBody interface:
FormBodyPart(String name, ContentBody body)
For example use: InputStreamBody:
new FormBodyPart("name", new InputStreamBody(new RandomInputStream(...)), ContentType.MULTIPART_FORM_DATA);
You can use: RandomInputStream class.
And with headers you could probably use: HTTPBuilder$RequestConfigDelegate.setHeaders(Map headers) because it's set to delegate of inner closure.
I've used curl in the past to do this testing:
curl -v -F "param1=1" -F "param2=99" -F "fileparam=#somefile.flv;type=video/x-flv" http://localhost:8080/someapp/sessions
somefile.flv is in the current directory
Related
I have a script groovy, this script for live fetching of docker image,
I want to add the authentication function with the private repository, but I am not familiar with groovy, who can help me, thanks
import groovy.json.JsonSlurper
// Set the URL we want to read from, it is MySQL from official Library for this example, limited to 20 results only.
docker_image_tags_url = "https://registry.adx.abc/v2/mysql/tags/list"
try {
// Set requirements for the HTTP GET request, you can add Content-Type headers and so on...
def http_client = new URL(docker_image_tags_url).openConnection() as HttpURLConnection
http_client.setRequestMethod('GET')
// Run the HTTP request
http_client.connect()
// Prepare a variable where we save parsed JSON as a HashMap, it's good for our use case, as we just need the 'name' of each tag.
def dockerhub_response = [:]
// Check if we got HTTP 200, otherwise exit
if (http_client.responseCode == 200) {
dockerhub_response = new JsonSlurper().parseText(http_client.inputStream.getText('UTF-8'))
} else {
println("HTTP response error")
System.exit(0)
}
// Prepare a List to collect the tag names into
def image_tag_list = []
// Iterate the HashMap of all Tags and grab only their "names" into our List
dockerhub_response.results.each { tag_metadata ->
image_tag_list.add(tag_metadata.name)
}
// The returned value MUST be a Groovy type of List or a related type (inherited from List)
// It is necessary for the Active Choice plugin to display results in a combo-box
return image_tag_list.sort()
} catch (Exception e) {
// handle exceptions like timeout, connection errors, etc.
println(e)
}
The problem has been resolved, thank you everyone for your help
// Import the JsonSlurper class to parse Dockerhub API response
import groovy.json.JsonSlurper
// Set the URL we want to read from, it is MySQL from official Library for this example, limited to 20 results only.
docker_image_tags_url = "https://registry.adx.vn/v2/form-be/tags/list"
try {
// Set requirements for the HTTP GET request, you can add Content-Type headers and so on...
def http_client = new URL(docker_image_tags_url).openConnection() as HttpURLConnection
http_client.setRequestMethod('GET')
String userCredentials = "your_user:your_passwd";
String basicAuth = "Basic " + new String(Base64.getEncoder().encode(userCredentials.getBytes()));
http_client.setRequestProperty ("Authorization", basicAuth);
// Run the HTTP request
http_client.connect()
// Prepare a variable where we save parsed JSON as a HashMap, it's good for our use case, as we just need the 'name' of each tag.
def dockerhub_response = [:]
// Check if we got HTTP 200, otherwise exit
if (http_client.responseCode == 200) {
dockerhub_response = new JsonSlurper().parseText(http_client.inputStream.getText('UTF-8'))
} else {
println("HTTP response error")
System.exit(0)
}
// Prepare a List to collect the tag names into
def image_tag_list = []
// Iterate the HashMap of all Tags and grab only their "names" into our List
dockerhub_response.tags.each { tag_metadata ->
image_tag_list.add(tag_metadata)
}
// The returned value MUST be a Groovy type of List or a related type (inherited from List)
// It is necessary for the Active Choice plugin to display results in a combo-box
return image_tag_list.sort()
} catch (Exception e) {
// handle exceptions like timeout, connection errors, etc.
println(e)
}
here is the result
I am now in trouble with the configuration variable GrailsApplication in my Integration Tests. I don't know why, but, I am not managing to get its value when testing my api. I am using Grails 3.3.11. The value of the variable is being null and, due to it, I can't authenticate to perform the tests. I would appreciate your help. I am using Grails 3.3.11.
package br.com.xxx.id.test.integration
//Imports were moved out to simplify understanding
class IdControllerSpec extends Specification {
def grailsApplication
#Value('${local.server.port}')
Integer serverPort
String accessToken
String baseUrl
JSONObject documentPropertiesForTesting
JSONObject documentForTesting
String partTest
String userTest
String typeIdTest
String refreshToken
void setup(){
baseUrl = "http://localhost:${serverPort}/cmbid/api/v1"
partTest = "partTest"
}
void "Saving a new and valid document properties"() {
when:
refreshToken = grailsApplication.config.getProperty('refreshToken')
accessToken = "Bearer " + authenticateXxxAut()
documentPropertiesForTesting = createNewTestDocumentProperties()
typeIdTest = documentPropertiesForTesting.get("message").toString().substring(20,52)
then:
documentPropertiesForTesting.get("status") == "ok"
documentPropertiesForTesting.get("message").contains("properly saved!")
cleanup:
DocumentProperties.withNewSession {
def dp = DocumentProperties.findById(typeIdTest)
dp.delete(flush: true)
}
}
def authenticateXxxAut() {
CloseableHttpClient httpClient = HttpClients.createDefault();
String response = ""
try {
JSONObject responseBody
println('****************************')
println(grailsApplication.config.getProperty('aut.newTokenUrl'))
println(grailsApplication.config.getProperty('refreshToken)'))
println('****************************')
def httpPost = new HttpPost(grailsApplication.config.getProperty('aut.newTokenUrl') + grailsApplication.config.getProperty('refreshToken)'))
CloseableHttpResponse httpResponse = httpClient.execute(httpPost)
if (httpResponse.getStatusLine().getStatusCode() == 200) {
responseBody = new JSONObject(EntityUtils.toString(httpResponse.getEntity()))
response = responseBody.get("access_token")
} else {
response = httpResponse.getStatusLine().getStatusCode().toString()
}
} catch (Exception e){
print(e.getLocalizedMessage())
} finally {
httpClient.close()
return response
}
}
I've been upgrading a Grails 2.x app to version 3.3.11 and just referencing the (provided) variable serverPort worked for me. The IDE shows it as being uninitialized but running the tests, it gets the correct value assigned. I also have my test classes annotated with #Integration(applicationClass = Application.class).
Here's how I get the URL to point against:
def url = "http://localhost:${serverPort}${grailsApplication.config.getProperty('server.contextPath', String, '')}"
i'm trying to generate swagger JSON files using https://github.com/pseudomuto/protoc-gen-doc, I can't find a way to exclude some of the APIs of the grpc service/fields inside the messages.
found the relevant styling in swagger, but can't seem to find a way to add it in the protobuf file http://watson-developer-cloud.github.io/api-guidelines/swagger-coding-style.html#excluding-operations-from-the-sdks
service MyService {
rpc ExternalApi (ExternalApiRequest) returns (ExternalApiResponse) {
option (google.api.http) = {
post: "/my/externalApi"
};
}
rpc InternalApi (InternalApiRequest) returns (InternalApiResponse) {
option (google.api.http) = {
post: "/my/internalApi"
};
}
message ExternalApiResponse {
string prefix = 1;
string id = 2; // field to exclude
}
// message to exclude
message Header { }
is there a way to exclude actions / fields from the protocol buffer files?
You can add
string id = 2 [(grpc.gateway.protoc_gen_swagger.options.openapiv2_field).read_only = true];
I want to upload a txt file to a website using a POST request with HTTPBuilder and multipart/form-data
I've tried running my function and I get a HTTP 200 OK response, but the file doesn't appear on the website anywhere.
private Map fileUpload(String url, File file){
log.debug "doPost: $url body: ${file.getName()}"
FileBody fileBody = new FileBody(file,ContentType.APPLICATION_OCTET_STREAM)
def result = [:]
try {
def authSite = new HTTPBuilder(url)
authSite.auth.basic(user, password)
authSite.request(POST) { req ->
headers.Accept = "application/json, text/javascript, */*; q=0.01"
req.params.setParameter(CoreConnectionPNames.SO_TIMEOUT, 20000)
req.params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 60000)
def mpe = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
mpe.addPart("gxt",fileBody)
req.setEntity(mpe)
response.success = { resp, reader ->
result = reader
}
response.failure = { resp, reader ->
println "My response handler got response: ${resp.statusLine}"
}
}
}
catch (e) {
log.debug("Could not perform POST request on URL $url", e)
throw e
}
return result
}
From debugging this is the status recieved
3695 [main] DEBUG org.apache.http.wire - << "HTTP/1.1 200 OK[\r][\n]"
3695 [main] DEBUG org.apache.http.wire - << "Date: Thu, 10 Jan 2019 07:34:06 GMT[\r][\n]"
Anything I'm doing wrong? I don't get any errors but it just seems like nothing happens.
I don't have anything conclusive, but I suspect there is something invalid with the way you set up the multipart upload.
To help figure this out, below is a standalone, working, multipart upload groovy script using HttpBuilder:
#Grab('org.codehaus.groovy.modules.http-builder:http-builder:0.7.1')
#Grab('org.apache.httpcomponents:httpmime:4.2.1')
import org.apache.http.entity.mime.content.*
import org.apache.http.entity.mime.*
import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.Method.POST
fileUpload('https://httpbin.org/post', new File('data.txt'))
Map fileUpload(String url, File file){
println "doPost: $url body: ${file.name}"
def result
try {
new HTTPBuilder(url).request(POST) { req ->
requestContentType = "multipart/form-data"
def content = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
content.addPart(file.name, new InputStreamBody(file.newInputStream(), file.name))
req.entity = content
// json might be something else (like a reader)
// depending on the response content type
response.success = { resp, json ->
result = json
println "RESP: ${resp.statusLine}, RESULT: $json"
}
response.failure = { resp, json ->
println "My response handler got response: ${resp.statusLine}"
}
}
} catch (e) {
println "Could not perform POST request on URL $url"
throw e
}
result
}
The script assumes a file data.txt with the data to post in the current directory. The script posts to httpbin.org as a working test endpoint, adjust accordingly to post to your endpoint instead.
Saving the above in test.groovy and executing will yield something like:
~> groovy test.groovy
doPost: https://httpbin.org/post body: data.txt
RESP: HTTP/1.1 200 OK, RESULT: [args:[:], data:, files:[data.txt:{ "foo": "bar" }], form:[:], headers:[Accept:*/*, Connection:close, Content-Type:multipart/form-data; boundary=ZVZuV5HAdPOt2Sv7ZjxuUHjd8sDAzCz9VkTqpJYP, Host:httpbin.org, Transfer-Encoding:chunked], json:null, origin:80.252.172.140, url:https://httpbin.org/post]
(note that first run will take a while as groovy grapes need to download the http-builder dependency tree)
perhaps starting with this working example and working your way back to your code would help you pinpoint whatever is not working in your code.
I am trying to use HTTP Builder to make a POST request in a pipeline script, (do have it in a shared lib where it works via the command line) but need it to work in Jenkins
I am getting the following error when running in Jenkins.
No suitable ClassLoader found for grab
My script looks as follows
#Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.7')
import static groovyx.net.http.ContentType.JSON
import static groovyx.net.http.Method.POST
import groovyx.net.http.HTTPBuilder
def gitUpdateStatus() {
String targetUrl = 'https://api.github.com/repos/myOrg/'
def http = new HTTPBuilder(targetUrl)
http.request(POST) {
uri.path = "myRepo/statuses/commit_id_here"
requestContentType = JSON
body = [state: 'failure', description: 'Jenkins Unit Tests', target_url: 'http://test.co.uk', context: 'unit tests']
headers.'Authorization' = "token 123"
headers.'User-Agent' = 'Jenkins Status Update'
headers.Accept = 'application/json'
response.success = { resp, json ->
println "GitHub updated successfully! ${resp.status}"
}
response.failure = { resp, json ->
println "GitHub update Failure! ${resp.status} " + json.message
}
}
node {
stage('Echo Client JS')
git branch: 'master', credentialsId: '${JENKINS_CREDENTIALS_ID}', url: 'git#github.com:myOrg/myRepo.git'
gitUpdateStatus()
}
I have seen many posts where the same issue has arisen but I just cannot seem to figure out how this has been fixed, can anyone assist please ?
Thank You
You cannot use #Grab directly in pipelines. You need to move gitUpdateStatus() function into Jenkins shared library. See Using third party libraries.