I support implemented tests framework and can't change version of tools, so need to resolve it with current version.
The test framework use RestAssered, Cucumber, Serenity.
When tests run in parallel sometimes they are failed due to several Authorization headers in request. In single run, everything is ok.
As I know Serenity just reuse features of the framework. So I think, maybe I configured in wrong way RestAssured.
My flow is:
In Before method I set up
#cucumber.api.java.Before
public void beforeScenario(Scenario scenario) {
RestAssured.baseURI = "https://test.net";
RestAssured.requestSpecification = new
Specification().requestSpecificationWithAdminToken(token);
RestAssured.responseSpecification = new Specification().responseSpecification();
RestAssured.config = RestAssuredConfig.config().decoderConfig(decoderConfig().noContentDecoders());
}
public class Specification {
public RequestSpecification requestSpecificationWithAdminToken(String token) {
return new RequestSpecBuilder().
addHeader("Authorization", "Bearer " + token).
setAccept(ContentType.JSON).
setContentType(ContentType.JSON).
log(LogDetail.ALL).
build();
}
public ResponseSpecification responseSpecification() {
return new ResponseSpecBuilder().
log(LogDetail.ALL).
build();
}
}
Tests like this
SomeBinding bindings = SerenityRest.
given().queryParams(params).
when().get("/api/some/GetEntities").
then().statusCode(HttpStatus.SC_OK).extract().response().as(SomeBinding.class);
For parallel run I use maven-failsafe-plugin with configuration
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<argLine>-Xmx1024m</argLine>
<includes>
<include>${test.include}</include>
</includes>
<excludes>
<exclude>${test.exclude}</exclude>
</excludes>
<parallel>classes</parallel>
<threadCount>${parallel.count}</threadCount>
<perCoreThreadCount>false</perCoreThreadCount>
<rerunFailingTestsCount>1</rerunFailingTestsCount>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
When I run tests in parallel I faced with issue that sometimes requests contain several Authorithation headers.
Authorization=Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1bmlxdWVfbmFtZSI6ImJjYzM3YjliOTM4ODQ3ODJiOWRhYjN
Authorization=Bearer OTM4ODQ3ODJiOWRhYjNmYjI5NDgyYzdjIiwianRpIjoiMjg3M2VjNDA0MmFjNGNkNzk0ODE5NTE0N2I5YzczMjUiL
Accept=application/json, application/javascript, text/javascript, text/json
Content-Type=application/json; charset=UTF-8
Could somebody please give advice how to resolve it?
Related
Context
I have a spring boot (version 2.2.6.RELEASE) web project.
From this web application (I call "APP1") I want to call another URI using the PATCH method from another web application (Let's call it "APP2").
In my pom.xml, I have the following dependency:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
Here is how I call the PATCH method of the other web application.
#FeignClient(name = "clientName", url = "base-uri")
public interface MyInterface{
#PatchMapping(value = "/target-uri")
void callClientMethod(Map<String, Object> args);
Problem
The APP2's PATCH method is effectively being called
But then APP1 throws the following error:
feign.RetryableException: Invalid HTTP method: PATCH executing PATCH
I looked on the Internet for a solution, and added the following snipet to my pom.xml
<dependency>
<groupId>com.netflix.feign</groupId> <!-- Also tried io.github.openfeign -->
<artifactId>feign-httpclient</artifactId>
<version>8.18.0</version>
</dependency>
After that, APP2's PATCH method is stille properly called but in APP1 I got the following error :
java.lang.NoSuchMethodError: feign.Response.create(ILjava/lang/String;Ljava/util/Map;Lfeign/Response$Body;)Lfeign/Response;
Question
Does anyone know how to solve this error ?
Thanks in advance for your help !
I had the same problem and spent a lot of time for understand and resolve this problem.
First what you need to understand that is the Feign doesn't support PATCH http method for call from the box!
And if you can change methods in both services use PUT for update instead PATCH...
But if you integrate with third party implementation you should add some configurations:
1. Add dependency which support PATCH http method:
// https://mvnrepository.com/artifact/io.github.openfeign/feign-okhttp
compile group: 'io.github.openfeign', name: 'feign-okhttp', version:
'10.2.0'
Add configuration:
#Configuration
public class FeignConfiguration {
#Bean
public OkHttpClient client() {
return new OkHttpClient();
}
}
And example for PATCH request with Feign:
#FeignClient(name = "someapi", url = "${client.someapi.url}")
#Component
#RequestMapping("/users")
public interface SomeClient {
#RequestMapping(value = "/{id}",
method = RequestMethod.PATCH,
consumes = MediaType.APPLICATION_JSON_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE)
FeignUser update(#PathVariable("id") Long id, #RequestBody Map<String, Object> fields);
}
Hope it helps someone.
Just Add:
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-httpclient</artifactId>
</dependency>
If you are adding Feign with the following dependency:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId> <!-- has dependecy on spring-cloud-openfeign-core inside, which already maintains version of feign-okhttp artifact -->
</dependency>
you can add okhttp client (without hardcoding artifact version) to fix the issue with PATCH request:
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-okhttp</artifactId> <!-- Required to use PATCH -->
</dependency>
No other steps needed. The okhttp client will be applied automatically by auto configuration.
Also, this way you don't need to manage feign-okhttp artifact version. Spring Cloud will manage version for you.
Tested with Spring Boot 2.7.6
The following config works for me:
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-jackson</artifactId>
<version>${feign.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-httpclient</artifactId>
<version>${feign.version}</version>
</dependency>
Where:
feign.version - 11.0
Spring Boot - 2.3.0.RELEASE
Spring-cloud.version - 2.2.3.RELEASE
I'm trying to setup Spring contract stub jar as either a fat jar with consumer or a http REST endpoint that my services can send request to and eventually receive the evaluated response.
Ideally I'd prefer the latter where consumer could run the stub to manage the interactions. My tests from consumer to local producer stubs work as expected. My compilation fails when I add the required annotations to the main class for stub runner. I think I'm missing some config or setup needed for the consumer to run the stub either as http REST endpoint or recognize within its m2.
The app compilation fails with #EnableStubRunnerServer complaining the below:
ConsumerApplication.java:[8,60] package org.springframework.cloud.contract.stubrunner.server does not exist
ConsumerApplication.java:[15,2] cannot find symbol
[ERROR] symbol: class EnableStubRunnerServer
ConsumerApplication.java:
#SpringBootApplication
#EnableWebMvc
#EnableStubRunnerServer
#Slf4j
public class ConsumerApplication {
public static void main(String[] args) {
SpringApplication.run(ConsumerApplication.class, args);
}
}
application.properties (Consumer):
stubrunner.ids=com.somecompany.somegroup:producer:0.0.1-SNAPSHOT:stubs:8081
stubrunner.stubsMode=REMOTE
server.ssl.key-store-password=password
server.ssl.key-password=password
server.ssl.trust-store-password=password
server.port=8081
Test.java: - This test works
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK)
#AutoConfigureMockMvc
#AutoConfigureJsonTesters
#AutoConfigureStubRunner( ids = "com.somecompany.somegroup:producer:+:stubs:8081",
stubsMode = StubRunnerProperties.StubsMode.LOCAL)
#DirtiesContext
public class ContractControllerTest extends AbstractTest {
}
pom:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-contract-stub-runner</artifactId>
<scope>test</scope>
</dependency>
Since you're trying to add test code to your main, production code, you have to change
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-contract-stub-runner</artifactId>
<scope>test</scope>
</dependency>
to
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-contract-stub-runner</artifactId>
</dependency>
I have a zuul gateway application, that receives requests from a client app and forwards the requests using load balanced rest template to a micro service with 2 endpoints, say endpoint1 and endpoint2 (load balancing between the two end points in round robbin which is okay for now, although I want it to be availability based).
Here are the issues I am facing -
I brought down one of the end points, say endpoint2 and tried calling the zuul route and I see that when the request is going to the endpoint2 - zuul takes 2 mins or so before failing with HTTP 503 and does not retry on the next request. the error is just cascaded back to the caller.
Also, even after setting the read time out and connect timeout configurations, I don't see ribbon respecting the configuration and still takes 2 mins to throw the error from the server.
I tried enabling logs at the netflix package level, but I am unable to see logs unless I pass a custom http client to rest template.
I am new to netflix stack of components... please advise if I am missing something obvious. Thanks
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycomp</groupId>
<artifactId>zuul-gateway</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>zuul-gateway</name>
<description>Spring Boot Zuul</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.9.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<spring-cloud.version>Edgware.SR1</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zuul</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-lambda</artifactId>
<version>1.11.242</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.3.10</version>
</dependency>
<dependency>
<groupId>com.netflix.netflix-commons</groupId>
<artifactId>netflix-commons-util</artifactId>
<version>0.1.1</version>
</dependency>
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
and my application.yml looks like below -
eureka:
client:
healthcheck:
enabled: true
lease:
duration: 5
service-url:
defaultZone: http://localhost:8761/eureka/
ingestWithOutEureka:
ribbon:
eureka:
enabled: false
NIWSServerListClassName: com.netflix.loadbalancer.ConfigurationBasedServerList
listOfServers: http://demo-nlb-6a67d59c901ecd128.elb.us-west-2.amazonaws.com,http://demo-nlb-124321w2a123ecd128.elb.us-west-2.amazonaws.com
okToRetryOnAllOperations: true
ConnectTimeout: 500
ReadTimeout: 1000
MaxAutoRetries: 5
MaxAutoRetriesNextServer: 5
MaxTotalHttpConnections: 500
MaxConnectionsPerHost: 100
retryableStatusCodes: 404,503
okhttp:
enabled: true
zuul:
debug:
request: true
parameter: true
ignored-services: '*'
routes:
ingestServiceELB:
path: /ingestWithoutEureka/ingest/**
retryable: true
url: http://dummyURL
management.security.enabled : false
spring:
application:
name: zuul-gateway
cloud:
loadbalancer:
retry:
enabled: true
logging:
level:
org:
apache:
http: DEBUG
com:
netflix: DEBUG
hystrix:
command:
default:
execution:
isolation:
strategy: THREAD
thread:
timeoutInMilliseconds: 60000
and my application class looks like below
#SpringBootApplication
#EnableZuulProxy
#EnableDiscoveryClient
public class ZuulGatewayApplication {
#Bean
public InterceptionFilter addInterceptionFilter() {
return new InterceptionFilter();
}
public static void main(String[] args) {
SpringApplication.run(ZuulGatewayApplication.class, args);
}
}
and lastly my zuul filter looks like below -
package com.zuulgateway.filter;
import com.netflix.zuul.ZuulFilter;
import com.netflix.zuul.context.RequestContext;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.cloud.client.loadbalancer.LoadBalancerClient;
import org.springframework.web.client.RestTemplate;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.stream.Collectors;
public class InterceptionFilter extends ZuulFilter{
private static final String REQUEST_PATH = "/ingestWithoutEureka";
#LoadBalanced
#Bean
RestTemplate loadBalanced() {
//RestTemplate restTemplate = new RestTemplate(new HttpComponentsClientHttpRequestFactory());
RestTemplate restTemplate = new RestTemplate();
return restTemplate;
}
#Autowired
#LoadBalanced
private RestTemplate loadBalancedRestTemplate;
#Override
public String filterType() {
return "route";
}
#Override
public int filterOrder() {
return 0;
}
#Override
public boolean shouldFilter() {
RequestContext context = RequestContext.getCurrentContext();
HttpServletRequest request = context.getRequest();
String method = request.getMethod();
String requestURI = request.getRequestURI();
return requestURI.startsWith(REQUEST_PATH);
}
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
try {
String requestPayload = ctx.getRequest().getReader().lines().collect(Collectors.joining(System.lineSeparator()));
String response = loadBalancedRestTemplate.postForObject("http://ingestWithOutEureka/ingest", requestPayload, String.class);
ctx.setResponseStatusCode(200);
ctx.setResponseBody(response);
} catch (IOException e) {
ctx.setResponseStatusCode(500);
ctx.setResponseBody("{ \"error\" : " + e.getMessage() + " }");
System.out.println("Exception during feign call - " + e.getMessage());
e.printStackTrace();
} finally {
ctx.setSendZuulResponse(false);
ctx.getResponse().setContentType("application/json");
}
return null;
}
}
So, here are the solutions that worked for me -
Issue 1 - Retry was not working in spite of configuring ribbon.<client>.OkToRetryOnAllOperations: true. Ribbon was clearly ignoring my configuration.
Solution: - It's strange, but after some debugging I had noticed that Ribbon was picking up client level configuration only if a global configuration was present in the first place.
Once I set the global "OkToRetryOnAllOperations" as either "true" or "false" as shown below, ribbon started picking up the ribbon.<client>.OkToRetryOnAllOperations as expected and I could see the retries happening.
ribbon:
OkToRetryOnAllOperations: false
Issue 2 - Also, even after setting the read time out and connect timeout configurations, I don't see ribbon respecting the configuration and still takes 2 mins to throw the error from the server
Solution 2 - Though ribbon started retrying requests after the changes suggested in solution 1 above, I did not see ribbon honoring <client>.ribbon.ReadTimeout and <client>.ribbon.ConnectTimeout.
After spending some time, I figure that this is because of using RestTemplate.
While spring documentation mentions that you could use load balanced RestTemplate for achieving retries, it does not mention that the timeouts wont work with it. Based on this SO answer from 2014, it looks like while ribbon has been added as a interceptor when using load balanced RestTemplate to achieve serviceId to URI resolution, ribbon does not use the underlying HTTP client and uses the http client provided by the RestTemplate. Thus, the ribbon specific <client>.ribbon.ReadTimeout and <client>.ribbon.ConnectTimeout are NOT honored. After I added timeouts to RestTemplate, requests have started timing out at expected intervals.
Lastly,
Issue 3 - I enabled logs by passing a custom http client to rest template.
#LoadBalanced
#Bean
RestTemplate loadBalanced() {
RestTemplate restTemplate = new RestTemplate(new HttpComponentsClientHttpRequestFactory());
System.out.println("returning load balanced rest client");
((HttpComponentsClientHttpRequestFactory)restTemplate.getRequestFactory()).setReadTimeout(1000*30);
((HttpComponentsClientHttpRequestFactory)restTemplate.getRequestFactory()).setConnectTimeout(1000*3);
((HttpComponentsClientHttpRequestFactory)restTemplate.getRequestFactory()).setConnectionRequestTimeout(1000*3);
return restTemplate;
}
#Bean
LoadBalancedBackOffPolicyFactory backOffPolicyFactory() {
return new LoadBalancedBackOffPolicyFactory() {
#Override
public BackOffPolicy createBackOffPolicy(String service) {
return new ExponentialBackOffPolicy();
}
};
}
With all the changes I see that the request retries are happening and with the timeouts and with exponential backoff and with request / responses logs visible as expected. Good luck!
I have a pretty simple question :)
According to feign documents, they are supporting in changing the basePath of a feign client object dynamically by passing URI parameter trough the api function like so:
GOOD Example:
interface MyClient {
#RequestLine("GET /internal-service")
String internalService(URI baseUrl);
}
The thing is I'm using swagger, who generates my API from a yaml file and adding #Param annotation to all function parameters, which is not good for me.
BAD Example:
interface MyClient {
#RequestLine("GET {baseUrl}/internal-service")
String internalService(#Param("baseUrl") String host);
}
Is there a way to make swagger generator to generate an API with a URI param, without the #Param annotation?
Desired Outcome Example:
interface MyClient {
#RequestLine("POST /internal-service")
String internalService(URI baseUrl, #Param("someParam") String someParam);
}
Initial solution:
After some investigation I came to realize that there is no support in swagger-codegen 2.2.3 for generating URI parameter as part of the client API function.
But there is an option in swagger-codegen-maven-plugin configuration of
"templateDirectory - directory with mustache templates", for giving a path to a folder including mustache templates, which will take the templates in that folder and override the existing with the same name.
example:
<plugin>
<groupId>io.swagger</groupId>
<artifactId>swagger-codegen-maven-plugin</artifactId>
<executions>
<execution>
<id>my-project-api-client-kit</id>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>${project.build.directory}/my-project-api.yaml</inputSpec>
<language>java</language>
<configOptions>
<dateLibrary>java8</dateLibrary>
<sourceFolder>src/main/java</sourceFolder>
</configOptions>
<modelPackage>my.project.ck.resources.models</modelPackage>
<apiPackage>my.project.ck.resources.interfaces</apiPackage>
<library>feign</library>
<templateDirectory>/myTemplateFolder/</templateDirectory>
</configuration>
</execution>
</executions>
</plugin>
And inside the custom templates folder, put your own api.mustache file with the additional "URI basePath" parameter:
...
{{#returnType}}{{{returnType}}} {{/returnType}}{{^returnType}}void {{/returnType}}{{nickname}}(URI basePath, {{#allParams}}{{^isBodyParam}}{{^legacyDates}}#Param("{{paramName}}") {{/legacyDates}}{{#legacyDates}}#Param(value="{{paramName}}", expander=ParamExpander.class) {{/legacyDates}}{{/isBodyParam}}{{{dataType}}} {{paramName}}{{#hasMore}}, {{/hasMore}}{{/allParams}});
...
Note:
Swagger provides a lot of templates for a variety of uses, make sure to take and modify the correct api.mustache template file according to which you are using.
in the above example I modified (and override) the java.libraries.feign/api.mustache file because this is the file my plugin is configured too (as in the example).
I'm develop a simple HTML5-Client for DFA-Reporting. For this I use a Google OAuth Service account, which works pretty fine on a client solution. But know I've been in trouble when I want to read the .p12-File within my Webapplication-Backend. Following problem appears:
java.io.IOException: DerInputStream.getLength(): lengthTag=111, too big.
at sun.security.util.DerInputStream.getLength(DerInputStream.java:561)
The application runs on an embedded Tomcat (Spring Boot) and I'm using the following approach the read to file:
File p12File = new java.io.File(this.getClass().getResource("/test-privatekey.p12").toURI());
this.httpTransport = GoogleNetHttpTransport.newTrustedTransport();
credential = new GoogleCredential.Builder().setTransport(httpTransport)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
.setServiceAccountScopes(SCOPES)
.setServiceAccountPrivateKeyFromP12File(p12File)
.build();
The Client-Solutions read the p12-File like this:
File p12File = new java.io.File("test-privatekey.p12");
Did anybody have a solution, or an solution-idea for this problem?
Many thanks and greetings from Berlin
Exclude p12 files in your POM.
<build>
...
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<configuration>
<nonFilteredFileExtensions>
<nonFilteredFileExtension>p12</nonFilteredFileExtension>
</nonFilteredFileExtensions>
</configuration>
</plugin>
</plugins>
</build>