I'm using springboot and rabbitmq to receive a message.
The first consumer i created works, declared as below:
#Component
public class UserConsumer {
#Autowired
private RabbitTemplate template;
#RabbitListener(queues = MessagingConfig.CONSUME_QUEUE)
public void consumeMessageFromQueue(MassTransitRequest userRequest) {
...
}
}
I then needed a second consumer so i duplicated the above and called it another name:
#Component
public class PackConsumer {
#Autowired
private RabbitTemplate template;
#RabbitListener(queues = MessagingConfig.CONSUME_QUEUE_CREATE_PACK)
public void consumeMessageFromQueue(MassTransitRequest fileRequest) {
...
}
}
Everything works locally on my machine, however when i deploy it the new queue does not process messages because there is no consumer connected to it. The UserConsumer continues to work.
Is there something else i should be doing in order to connect to the new queue at the same time as the original?
During my learning i did add a "MessagingConfig" class as below, however i believe it relates to sending messages and not receiving them or an alternative configuration:
#Configuration
public class MessagingConfig {
public static final String CONSUME_QUEUE = "merge-document-request";
public static final String CONSUME_EXCHANGE = "merge-document-request";
public static final String CONSUME_ROUTING_KEY = "";
public static final String PUBLISH_QUEUE = "merge-document-response";
public static final String PUBLISH_EXCHANGE = "merge-document-response";
public static final String PUBLISH_ROUTING_KEY = "";
public static final String CONSUME_QUEUE_CREATE_PACK = "create-pack-request";
public static final String CONSUME_EXCHANGE_CREATE_PACK = "create-pack-request";
public static final String CONSUME_ROUTING_KEY_CREATE_PACK = "";
public static final String PUBLISH_QUEUE_CREATE_PACK = "create-pack-response";
public static final String PUBLISH_EXCHANGE_CREATE_PACK = "create-pack-response";
public static final String PUBLISH_ROUTING_KEY_CREATE_PACK = "";
#Bean
public Queue queue() {
return new Queue(CONSUME_QUEUE);
}
#Bean
public TopicExchange exchange() {
return new TopicExchange(CONSUME_EXCHANGE);
}
#Bean
public Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(CONSUME_ROUTING_KEY);
}
#Bean
public MessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public AmqpTemplate template(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(converter());
return rabbitTemplate;
}
}
Thanks in advance
public class MyApplication extends Application<MyConfiguration> {
final static Logger LOG = Logger.getLogger(MyApplication.class);
public static void main(final String[] args) throws Exception {
new MyApplication().run(args);
}
#Override
public String getName() {
return "PFed";
}
#Override
public void initialize(final Bootstrap<MyConfiguration> bootstrap) {
// TODO: application initialization
bootstrap.addBundle(new DBIExceptionsBundle());
}
#Override
public void run(final MyConfiguration configuration,
final Environment environment) {
// TODO: implement application
final DBIFactory factory = new DBIFactory();
final DBI jdbi = factory.build(environment, configuration.getDataSourceFactory(), "postgresql");
UserDAO userDAO = jdbi.onDemand(UserDAO.class);
userDAO.findNameById(1);
UserResource userResource = new UserResource(new UserService(userDAO));
environment.jersey().register(userResource);
}
I get the the following error at findNameById.
java.lang.NoSuchMethodError: java.lang.Object.findNameById(I)Ljava/lang/String;
at org.skife.jdbi.v2.sqlobject.CloseInternalDoNotUseThisClass$$EnhancerByCGLIB$$a0e63670.CGLIB$findNameById$5()
}
public interface UserDAO {
#SqlQuery("select userId from user where id = :email")
User isEmailAndUsernameUnique(#Bind("email") String email);
#SqlQuery("select name from something where id = :id")
String findNameById(#Bind("id") int id);
}
i have a processor which should produce kstream JMX metrics:
public class ProcessorJMX implements Processor<String, GenericRecord> {
private StreamsMetrics streamsMetrics;
private Sensor sensorStartTs;
#Override
public void init(ProcessorContext processorContext) {
streamsMetrics = processorContext.metrics();
sensorStartTs = streamsMetrics.addSensor("start_ts", Sensor.RecordingLevel.INFO);
}
#Override
public void process(String key, GenericRecord val) {
streamsMetrics.recordThroughput(sensorStartTs, Long.valueOf(val.get("start_ts").toString()));
}
#Override
public void punctuate(long l) { }
#Override
public void close() { }
}
then i use this on my output topic and start my integration test. but when i look in jconsole, i dont see this metric anywhere. where can i find it in jconsole under MBeans?
do i have to do something else before it becomes visible?
here are the properties i am using:
Properties testProperties = new Properties();
testProperties.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG,
CLUSTER.bootstrapServers());
testProperties.put("confluent.metrics.reporter.bootstrap.servers", CLUSTER.bootstrapServers());
testProperties.put("metrics.recording.level", "DEBUG");
testProperties.put("metric.reporters", "org.apache.kafka.common.metrics.JmxReporter");
what is wrong with this config?
The following is what I added to the init:
#Override
public void init(ProcessorContext processorContext) {
streamsMetrics = processorContext.metrics();
Map<String, String> metricTags = new HashMap<String, String>();
metricTags.put("metricTagKey", "metricsTagVal");
MetricConfig metricConfig = new MetricConfig().tags(metricTags);
Metrics metrics = new Metrics(metricConfig);
sensorStartTs = metrics.sensor("start_ts");
MetricName metricName = metrics.metricName("x-name", "x-group", "x-description");
sensorStartTs = streamsMetrics.addSensor("start_ts", Sensor.RecordingLevel.INFO);
sensorStartTs.add(metricName, new Min());
}
This MetricName class helped.
As I'm using ResponseEntity<T> as return value for my FeignClient method, I was expecting it to return a ResponseEntity with 400 status if it's what the server returns. But instead it throws a FeignException.
How can I get a proper ResponseEntity instead of an Exception from FeignClient ?
Here is my FeignClient:
#FeignClient(value = "uaa", configuration = OauthFeignClient.Conf.class)
public interface OauthFeignClient {
#RequestMapping(
value = "/oauth/token",
method = RequestMethod.POST,
consumes = MULTIPART_FORM_DATA_VALUE,
produces = APPLICATION_JSON_VALUE)
ResponseEntity<OauthTokenResponse> token(Map<String, ?> formParams);
class Conf {
#Value("${oauth.client.password}")
String oauthClientPassword;
#Bean
public Encoder feignFormEncoder() {
return new SpringFormEncoder();
}
#Bean
public Contract feignContract() {
return new SpringMvcContract();
}
#Bean
public BasicAuthRequestInterceptor basicAuthRequestInterceptor() {
return new BasicAuthRequestInterceptor("web-client", oauthClientPassword);
}
}
}
and here how I use it:
#PostMapping("/login")
public ResponseEntity<LoginTokenPair> getTokens(#RequestBody #Valid LoginRequest userCredentials) {
Map<String, String> formData = new HashMap<>();
ResponseEntity<OauthTokenResponse> response = oauthFeignClient.token(formData);
//code never reached if contacted service returns a 400
...
}
By the way, solution I gave before works, but my initial intention is bad idea: an error is an error and should not be handled on nominal flow. Throwing an exception, like Feign does, and handling it with an #ExceptionHandler is a better way to go in Spring MVC world.
So two solutions:
add an #ExceptionHandler for FeignException
configure the FeignClient with an ErrorDecoder to translate the error in an Exception your business layer knows about (and already provide #ExceptionHandler for)
I prefer second solution because received error message structure is likely to change from a client to an other, so you can extract finer grained data from those error with a per-client error decoding.
FeignClient with conf (sorry for the noise introduced by feign-form)
#FeignClient(value = "uaa", configuration = OauthFeignClient.Config.class)
public interface OauthFeignClient {
#RequestMapping(
value = "/oauth/token",
method = RequestMethod.POST,
consumes = MULTIPART_FORM_DATA_VALUE,
produces = APPLICATION_JSON_VALUE)
DefaultOAuth2AccessToken token(Map<String, ?> formParams);
#Configuration
class Config {
#Value("${oauth.client.password}")
String oauthClientPassword;
#Autowired
private ObjectFactory<HttpMessageConverters> messageConverters;
#Bean
public Encoder feignFormEncoder() {
return new SpringFormEncoder(new SpringEncoder(messageConverters));
}
#Bean
public Decoder springDecoder() {
return new ResponseEntityDecoder(new SpringDecoder(messageConverters));
}
#Bean
public Contract feignContract() {
return new SpringMvcContract();
}
#Bean
public BasicAuthRequestInterceptor basicAuthRequestInterceptor() {
return new BasicAuthRequestInterceptor("web-client", oauthClientPassword);
}
#Bean
public ErrorDecoder uaaErrorDecoder(Decoder decoder) {
return (methodKey, response) -> {
try {
OAuth2Exception uaaException = (OAuth2Exception) decoder.decode(response, OAuth2Exception.class);
return new SroException(
uaaException.getHttpErrorCode(),
uaaException.getOAuth2ErrorCode(),
Arrays.asList(uaaException.getSummary()));
} catch (Exception e) {
return new SroException(
response.status(),
"Authorization server responded with " + response.status() + " but failed to parse error payload",
Arrays.asList(e.getMessage()));
}
};
}
}
}
Common business exception
public class SroException extends RuntimeException implements Serializable {
public final int status;
public final List<String> errors;
public SroException(final int status, final String message, final Collection<String> errors) {
super(message);
this.status = status;
this.errors = Collections.unmodifiableList(new ArrayList<>(errors));
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof SroException)) return false;
SroException sroException = (SroException) o;
return status == sroException.status &&
Objects.equals(super.getMessage(), sroException.getMessage()) &&
Objects.equals(errors, sroException.errors);
}
#Override
public int hashCode() {
return Objects.hash(status, super.getMessage(), errors);
}
}
Error handler (extracted from a ResponseEntityExceptionHandler extension)
#ExceptionHandler({SroException.class})
public ResponseEntity<Object> handleSroException(SroException ex) {
return new SroError(ex).toResponse();
}
Error response DTO
#XmlRootElement
public class SroError implements Serializable {
public final int status;
public final String message;
public final List<String> errors;
public SroError(final int status, final String message, final Collection<String> errors) {
this.status = status;
this.message = message;
this.errors = Collections.unmodifiableList(new ArrayList<>(errors));
}
public SroError(final SroException e) {
this.status = e.status;
this.message = e.getMessage();
this.errors = Collections.unmodifiableList(e.errors);
}
protected SroError() {
this.status = -1;
this.message = null;
this.errors = null;
}
public ResponseEntity<Object> toResponse() {
return new ResponseEntity(this, HttpStatus.valueOf(this.status));
}
public ResponseEntity<Object> toResponse(HttpHeaders headers) {
return new ResponseEntity(this, headers, HttpStatus.valueOf(this.status));
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof SroError)) return false;
SroError sroException = (SroError) o;
return status == sroException.status &&
Objects.equals(message, sroException.message) &&
Objects.equals(errors, sroException.errors);
}
#Override
public int hashCode() {
return Objects.hash(status, message, errors);
}
}
Feign client usage notice how errors are transparently handled (no try / catch) thanks to #ControllerAdvice & #ExceptionHandler({SroException.class})
#RestController
#RequestMapping("/uaa")
public class AuthenticationController {
private static final BearerToken REVOCATION_TOKEN = new BearerToken("", 0L);
private final OauthFeignClient oauthFeignClient;
private final int refreshTokenValidity;
#Autowired
public AuthenticationController(
OauthFeignClient oauthFeignClient,
#Value("${oauth.ttl.refresh-token}") int refreshTokenValidity) {
this.oauthFeignClient = oauthFeignClient;
this.refreshTokenValidity = refreshTokenValidity;
}
#PostMapping("/login")
public ResponseEntity<LoginTokenPair> getTokens(#RequestBody #Valid LoginRequest userCredentials) {
Map<String, String> formData = new HashMap<>();
formData.put("grant_type", "password");
formData.put("client_id", "web-client");
formData.put("username", userCredentials.username);
formData.put("password", userCredentials.password);
formData.put("scope", "openid");
DefaultOAuth2AccessToken response = oauthFeignClient.token(formData);
return ResponseEntity.ok(new LoginTokenPair(
new BearerToken(response.getValue(), response.getExpiresIn()),
new BearerToken(response.getRefreshToken().getValue(), refreshTokenValidity)));
}
#PostMapping("/logout")
public ResponseEntity<LoginTokenPair> revokeTokens() {
return ResponseEntity
.ok(new LoginTokenPair(REVOCATION_TOKEN, REVOCATION_TOKEN));
}
#PostMapping("/refresh")
public ResponseEntity<BearerToken> refreshToken(#RequestHeader("refresh_token") String refresh_token) {
Map<String, String> formData = new HashMap<>();
formData.put("grant_type", "refresh_token");
formData.put("client_id", "web-client");
formData.put("refresh_token", refresh_token);
formData.put("scope", "openid");
DefaultOAuth2AccessToken response = oauthFeignClient.token(formData);
return ResponseEntity.ok(new BearerToken(response.getValue(), response.getExpiresIn()));
}
}
So, looking at source code, it seams that only solution is actually using feign.Response as return type for FeignClient methods and hand decoding the body with something like new ObjectMapper().readValue(response.body().asReader(), clazz) (with a guard on 2xx status of course because for error statuses, it's very likely that body is an error description and not a valid payload ;).
This makes possible to extract and forward status, header, body, etc. even if status is not in 2xx range.
Edit:
Here is a way to forward status, headers and mapped JSON body (if possible):
public static class JsonFeignResponseHelper {
private final ObjectMapper json = new ObjectMapper();
public <T> Optional<T> decode(Response response, Class<T> clazz) {
if(response.status() >= 200 && response.status() < 300) {
try {
return Optional.of(json.readValue(response.body().asReader(), clazz));
} catch(IOException e) {
return Optional.empty();
}
} else {
return Optional.empty();
}
}
public <T, U> ResponseEntity<U> toResponseEntity(Response response, Class<T> clazz, Function<? super T, ? extends U> mapper) {
Optional<U> payload = decode(response, clazz).map(mapper);
return new ResponseEntity(
payload.orElse(null),//didn't find a way to feed body with original content if payload is empty
convertHeaders(response.headers()),
HttpStatus.valueOf(response.status()));
}
public MultiValueMap<String, String> convertHeaders(Map<String, Collection<String>> responseHeaders) {
MultiValueMap<String, String> responseEntityHeaders = new LinkedMultiValueMap<>();
responseHeaders.entrySet().stream().forEach(e ->
responseEntityHeaders.put(e.getKey(), new ArrayList<>(e.getValue())));
return responseEntityHeaders;
}
}
that can be used as follow:
#PostMapping("/login")
public ResponseEntity<LoginTokenPair> getTokens(#RequestBody #Valid LoginRequest userCredentials) throws IOException {
Response response = oauthFeignClient.token();
return feignHelper.toResponseEntity(
response,
OauthTokenResponse.class,
oauthTokenResponse -> new LoginTokenPair(
new BearerToken(oauthTokenResponse.access_token, oauthTokenResponse.expires_in),
new BearerToken(oauthTokenResponse.refresh_token, refreshTokenValidity)));
}
This saves headers and status code, but error message is lost :/
I'd like to not require my clients to provide content_type application/json but just default to it. I got this working.
I also tried to combine with another example to implement a custom isFatal(Throwable t) from ConditionalRejectingErrorHandler. I can get my custom error handler to fire, but then it seems to require the content_type property again. I can't figure out how to get them both to work at the same time.
Any ideas?
The below successfully works to not require content_type
EDIT: The below code does not work as I thought. An old message in the queue with the property content_type application/json must have been pulled in
#EnableRabbit
#Configuration
public class ExampleRabbitConfigurer implements
RabbitListenerConfigurer {
#Value("${spring.rabbitmq.host:'localhost'}")
private String host;
#Value("${spring.rabbitmq.port:5672}")
private int port;
#Value("${spring.rabbitmq.username}")
private String username;
#Value("${spring.rabbitmq.password}")
private String password;
#Autowired
private MappingJackson2MessageConverter mappingJackson2MessageConverter;
#Autowired
private DefaultMessageHandlerMethodFactory messageHandlerMethodFactory;
#Bean
public MappingJackson2MessageConverter mappingJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(mappingJackson2MessageConverter);
return factory;
}
#Override
public void configureRabbitListeners(final RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory);
}
The below here works to override isFatal() in ConditionalRejectingErrorHandler. The SimpleRabbitListenerContainerFactory.setMessageConverter() seems like it should serve the same purpose as DefaultMessageHandlerMethodFactory.setMessageConverter(). Obviously this is not the case.
#EnableRabbit
#Configuration
public class ExampleRabbitConfigurer {
#Value("${spring.rabbitmq.host:'localhost'}")
private String host;
#Value("${spring.rabbitmq.port:5672}")
private int port;
#Value("${spring.rabbitmq.username}")
private String username;
#Value("${spring.rabbitmq.password}")
private String password;
#Autowired
ConnectionFactory connectionFactory;
#Autowired
Jackson2JsonMessageConverter jackson2JsonConverter;
#Autowired
ErrorHandler amqpErrorHandlingExceptionStrategy;
#Bean
public Jackson2JsonMessageConverter jackson2JsonConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public ErrorHandler amqpErrorHandlingExceptionStrategy() {
return new ConditionalRejectingErrorHandler(new AmqpErrorHandlingExceptionStrategy());
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(jackson2JsonConverter);
factory.setErrorHandler(amqpErrorHandlingExceptionStrategy);
return factory;
}
public static class AmqpErrorHandlingExceptionStrategy extends ConditionalRejectingErrorHandler.DefaultExceptionStrategy {
private final Logger LOGGER = org.slf4j.LoggerFactory.getLogger(getClass());
#Override
public boolean isFatal(Throwable t) {
if (t instanceof ListenerExecutionFailedException) {
ListenerExecutionFailedException lefe = (ListenerExecutionFailedException) t;
LOGGER.error("Failed to process inbound message from queue "
+ lefe.getFailedMessage().getMessageProperties().getConsumerQueue()
+ "; failed message: " + lefe.getFailedMessage(), t);
}
return super.isFatal(t);
}
}
Use an "after receive" MessagePostProcessor to add the contentType header to the inbound message.
Starting with version 2.0, you can add the MPP to the container factory.
For earlier versions you can reconfigure...
#SpringBootApplication
public class So47424449Application {
public static void main(String[] args) {
SpringApplication.run(So47424449Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitListenerEndpointRegistry registry, RabbitTemplate template) {
return args -> {
SimpleMessageListenerContainer container =
(SimpleMessageListenerContainer) registry.getListenerContainer("myListener");
container.setAfterReceivePostProcessors(m -> {
m.getMessageProperties().setContentType("application/json");
return m;
});
container.start();
// send a message with no content type
template.setMessageConverter(new SimpleMessageConverter());
template.convertAndSend("foo", "{\"bar\":\"baz\"}", m -> {
m.getMessageProperties().setContentType(null);
return m;
});
template.convertAndSend("foo", "{\"bar\":\"qux\"}", m -> {
m.getMessageProperties().setContentType(null);
return m;
});
};
}
#Bean
public Jackson2JsonMessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
#RabbitListener(id = "myListener", queues = "foo", autoStartup = "false")
public void listen(Foo foo) {
System.out.println(foo);
if (foo.bar.equals("qux")) {
throw new MessageConversionException("test");
}
}
public static class Foo {
public String bar;
public String getBar() {
return this.bar;
}
public void setBar(String bar) {
this.bar = bar;
}
#Override
public String toString() {
return "Foo [bar=" + this.bar + "]";
}
}
}
As you can see, since it modifies the source message, the modified header is available in the error handler...
2017-11-22 09:39:26.615 WARN 97368 --- [cTaskExecutor-1] ingErrorHandler$DefaultExceptionStrategy : Fatal message conversion error; message rejected; it will be dropped or routed to a dead letter exchange, if so configured: (Body:'{"bar":"qux"}' MessageProperties [headers={}, contentType=application/json, contentEncoding=UTF-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=, receivedRoutingKey=foo, deliveryTag=2, consumerTag=amq.ctag-re1kcxKV14L_nl186stM0w, consumerQueue=foo]), contentType=application/json, contentEncoding=UTF-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=, receivedRoutingKey=foo, deliveryTag=2, consumerTag=amq.ctag-re1kcxKV14L_nl186stM0w, consumerQueue=foo])