I have a Micronaut application that uses Micrometer to report metrics to InfluxDB with the micronaut-micrometer project. Currently it is using the Statsd Registry provided via the io.micronaut.configuration:micronaut-micrometer-registry-statsd dependency.
I would like to instead output metrics in Influx Line Protocol (ILP), but the micronaut-micrometer project does not offer an Influx Registry currently. I tried to work around this by importing the io.micrometer:micrometer-registry-influx dependency and configuring an InfluxMeterRegistry manually like this:
#Factory
public class MyMetricRegistryConfigurer implements MeterRegistryConfigurer {
#Bean
#Primary
#Singleton
public MeterRegistry getMeterRegistry() {
InfluxConfig config = new InfluxConfig() {
#Override
public Duration step() {
return Duration.ofSeconds(10);
}
#Override
public String db() {
return "metrics";
}
#Override
public String get(String k) {
return null; // accept the rest of the defaults
}
};
return new InfluxMeterRegistry(config, Clock.SYSTEM);
}
#Override
public boolean supports(MeterRegistry meterRegistry) {
return meterRegistry instanceof InfluxMeterRegistry;
}
}
When the application runs, the metrics are exposed on my /metrics endpoint as I would expect, but nothing gets written to InfluxDB. I confirmed that my local InfluxDB accepts metrics at the expected localhost:8086/write?db=metrics endpoint using curl. Can anyone give me some pointers to get this working? I'm wondering if I need to manually define a reporter somewhere...
After playing around for a bit, I got this working with the following code:
#Factory
public class InfluxMeterRegistryFactory {
#Bean
#Singleton
#Requires(property = MeterRegistryFactory.MICRONAUT_METRICS_ENABLED, value =
StringUtils.TRUE, defaultValue = StringUtils.TRUE)
#Requires(beans = CompositeMeterRegistry.class)
public InfluxMeterRegistry getMeterRegistry() {
InfluxConfig config = new InfluxConfig() {
#Override
public Duration step() {
return Duration.ofSeconds(10);
}
#Override
public String db() {
return "metrics";
}
#Override
public String get(String k) {
return null; // accept the rest of the defaults
}
};
return new InfluxMeterRegistry(config, Clock.SYSTEM);
}
}
I also noticed that an InfluxMeterRegistry will be available out of the box in the future for micronaut-micrometer as of v1.2.0.
Related
I'm trying to rewrite some legacy code which used org.springframework.security.oauth2.provider.token.store.InMemoryTokenStore to store the access tokens. I'm currently trying to use the RedisTokenStore instead of the previously used InMemoryTokenStore. The token gets generated and gets stored in Redis fine (Standalone redis configuration), however, deserialization of OAuth2Authentication fails with the following error:
Could not read JSON: Cannot construct instance of `org.springframework.security.oauth2.provider.OAuth2Authentication` (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
Since there's no default constructor for this class, the deserialization and mapping to the actual object while looking up from Redis fails.
RedisTokenStore redisTokenStore = new RedisTokenStore(jedisConnectionFactory);
redisTokenStore.setSerializationStrategy(new StandardStringSerializationStrategy() {
#Override
protected <T> T deserializeInternal(byte[] bytes, Class<T> aClass) {
return Utilities.parse(new String(bytes, StandardCharsets.UTF_8),aClass);
}
#Override
protected byte[] serializeInternal(Object o) {
return Objects.requireNonNull(Utilities.convert(o)).getBytes();
}
});
this.tokenStore = redisTokenStore;
public static <T> T parse(String json, Class<T> clazz) {
try {
return OBJECT_MAPPER.readValue(json, clazz);
} catch (IOException e) {
log.error("Jackson2Json failed: " + e.getMessage());
} return null;}
public static String convert(Object data) {
try {
return OBJECT_MAPPER.writeValueAsString(data);
} catch (JsonProcessingException e) {
log.error("Conversion failed: " + e.getMessage());
}
return null;
}
How is OAuth2Authentication object reconstructed when the token is looked up from Redis? Since it does not define a default constructor, any Jackson based serializer and object mapper won't be able to deserialize it.
Again, the serialization works great (since OAuth2Authentication implements Serializable interface) and the token gets stored fine in Redis. It just fails when the /oauth/check_token is called.
What am I missing and how is this problem dealt with while storing access token in Redis?
I solved the issue by writing custom deserializer. It looks like this:
import com.fasterxml.jackson.core.JacksonException;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;
import org.springframework.security.oauth2.core.AuthorizationGrantType;
import java.io.IOException;
public class AuthorizationGrantTypeCustomDeserializer extends JsonDeserializer<AuthorizationGrantType> {
#Override
public AuthorizationGrantType deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JacksonException {
Root root = p.readValueAs(Root.class);
return root != null ? new AuthorizationGrantType(root.value) : new AuthorizationGrantType("");
}
private static class Root {
public String value;
}
public static SimpleModule generateModule() {
SimpleModule authGrantModule = new SimpleModule();
authGrantModule.addDeserializer(AuthorizationGrantType.class, new AuthorizationGrantTypeCustomDeserializer());
return authGrantModule;
}
}
Then I registered deserializer in objectMapper which is later used by jackson API
ObjectMapper mapper = new ObjectMapper()
.registerModule(AuthorizationGrantTypeCustomDeserializer.generateModule());
Little background: I am working on a topology using Apache Storm, I thought why not use dependency injection in it, but I was not sure how it will behave on cluster environment when topology deployed to cluster. I started looking for answers on if DI is good option to use in Storm topologies, I came across some threads about Apache Spark where it was mentioned serialization is going to be problem and saw some responses for apache storm along the same lines. So finally I decided to write a sample topology with google guice to see what happens.
I wrote a sample topology with two bolts, and used google guice to injects dependencies. First bolt emits a tick tuple, then first bolt creates message, bolt prints the message on log and call some classes which does the same. Then this message is emitted to second bolt and same printing logic there as well.
First Bolt
public class FirstBolt extends BaseRichBolt {
private OutputCollector collector;
private static int count = 0;
private FirstInjectClass firstInjectClass;
#Override
public void prepare(Map map, TopologyContext topologyContext, OutputCollector outputCollector) {
collector = outputCollector;
Injector injector = Guice.createInjector(new Module());
firstInjectClass = injector.getInstance(FirstInjectClass.class);
}
#Override
public void execute(Tuple tuple) {
count++;
String message = "Message count "+count;
firstInjectClass.printMessage(message);
log.error(message);
collector.emit("TO_SECOND_BOLT", new Values(message));
collector.ack(tuple);
}
#Override
public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
outputFieldsDeclarer.declareStream("TO_SECOND_BOLT", new Fields("MESSAGE"));
}
#Override
public Map<String, Object> getComponentConfiguration() {
Config conf = new Config();
conf.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 10);
return conf;
}
}
Second Bolt
public class SecondBolt extends BaseRichBolt {
private OutputCollector collector;
private SecondInjectClass secondInjectClass;
#Override
public void prepare(Map map, TopologyContext topologyContext, OutputCollector outputCollector) {
collector = outputCollector;
Injector injector = Guice.createInjector(new Module());
secondInjectClass = injector.getInstance(SecondInjectClass.class);
}
#Override
public void execute(Tuple tuple) {
String message = (String) tuple.getValue(0);
secondInjectClass.printMessage(message);
log.error("SecondBolt {}",message);
collector.ack(tuple);
}
#Override
public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
}
}
Class in which dependencies are injected
public class FirstInjectClass {
FirstInterface firstInterface;
private final String prepend = "FirstInjectClass";
#Inject
public FirstInjectClass(FirstInterface firstInterface) {
this.firstInterface = firstInterface;
}
public void printMessage(String message){
log.error("{} {}", prepend, message);
firstInterface.printMethod(message);
}
}
Interface used for binding
public interface FirstInterface {
void printMethod(String message);
}
Implementation of interface
public class FirstInterfaceImpl implements FirstInterface{
private final String prepend = "FirstInterfaceImpl";
public void printMethod(String message){
log.error("{} {}", prepend, message);
}
}
Same way another class that receives dependency via DI
public class SecondInjectClass {
SecondInterface secondInterface;
private final String prepend = "SecondInjectClass";
#Inject
public SecondInjectClass(SecondInterface secondInterface) {
this.secondInterface = secondInterface;
}
public void printMessage(String message){
log.error("{} {}", prepend, message);
secondInterface.printMethod(message);
}
}
another interface for binding
public interface SecondInterface {
void printMethod(String message);
}
implementation of second interface
public class SecondInterfaceImpl implements SecondInterface{
private final String prepend = "SecondInterfaceImpl";
public void printMethod(String message){
log.error("{} {}", prepend, message);
}
}
Module Class
public class Module extends AbstractModule {
#Override
protected void configure() {
bind(FirstInterface.class).to(FirstInterfaceImpl.class);
bind(SecondInterface.class).to(SecondInterfaceImpl.class);
}
}
Nothing fancy here, just two bolts and couple of classes for DI. I deployed it on server and it works just fine. The catch/problem though is that I have to initialize Injector in each bolt which makes me question what is side effect of it going to be?
This implementation is simple, just 2 bolts.. what if I have more bolts? what impact it would create on topology if I have to initialize Injector in all bolts?
If I try to initialize Injector outside prepare method I get error for serialization.
Say my users subscribe to a plan. Is it possible then using Spring Cloud Gateway to rate limit user requests based up on the subscription plan? Given there're Silver and Gold plans, would it let Silver subscriptions to have replenishRate/burstCapacity of 5/10 and Gold 50/100?
I naively thought of passing a new instance of RedisRateLimiter (see below I construct a new one with 5/10 settings) to the filter but I needed to get the information about the user from the request somehow in order to be able to find out whether it is Silver and Gold plan.
#Bean
public RouteLocator myRoutes(RouteLocatorBuilder builder) {
return builder.routes()
.route(p -> p
.path("/get")
.filters(f ->
f.requestRateLimiter(r -> {
r.setRateLimiter(new RedisRateLimiter(5, 10))
})
.uri("http://httpbin.org:80"))
.build();
}
Am I trying to achieve something that is even possible with Spring Cloud Gateway? What other products would you recommend to check for the purpose if any?
Thanks!
Okay, it is possible by creating a custom rate limiter on top of RedisRateLimiter class. Unfortunately the class has not been architected for extendability so the solution is somewhat "hacky", I could only decorate the normal RedisRateLimiter and duplicate some of its code in there:
#Primary
#Component
public class ApiKeyRateLimiter implements RateLimiter {
private Log log = LogFactory.getLog(getClass());
// How many requests per second do you want a user to be allowed to do?
private static final int REPLENISH_RATE = 1;
// How much bursting do you want to allow?
private static final int BURST_CAPACITY = 1;
private final RedisRateLimiter rateLimiter;
private final RedisScript<List<Long>> script;
private final ReactiveRedisTemplate<String, String> redisTemplate;
#Autowired
public ApiKeyRateLimiter(
RedisRateLimiter rateLimiter,
#Qualifier(RedisRateLimiter.REDIS_SCRIPT_NAME) RedisScript<List<Long>> script,
ReactiveRedisTemplate<String, String> redisTemplate) {
this.rateLimiter = rateLimiter;
this.script = script;
this.redisTemplate = redisTemplate;
}
// These two methods are the core of the rate limiter
// Their purpose is to come up with a rate limits for given API KEY (or user ID)
// It is up to implementor to return limits based up on the api key passed
private int getBurstCapacity(String routeId, String apiKey) {
return BURST_CAPACITY;
}
private int getReplenishRate(String routeId, String apiKey) {
return REPLENISH_RATE;
}
public Mono<Response> isAllowed(String routeId, String apiKey) {
int replenishRate = getReplenishRate(routeId, apiKey);
int burstCapacity = getBurstCapacity(routeId, apiKey);
try {
List<String> keys = getKeys(apiKey);
// The arguments to the LUA script. time() returns unixtime in seconds.
List<String> scriptArgs = Arrays.asList(replenishRate + "", burstCapacity + "",
Instant.now().getEpochSecond() + "", "1");
Flux<List<Long>> flux = this.redisTemplate.execute(this.script, keys, scriptArgs);
return flux.onErrorResume(throwable -> Flux.just(Arrays.asList(1L, -1L)))
.reduce(new ArrayList<Long>(), (longs, l) -> {
longs.addAll(l);
return longs;
}) .map(results -> {
boolean allowed = results.get(0) == 1L;
Long tokensLeft = results.get(1);
Response response = new Response(allowed, getHeaders(tokensLeft, replenishRate, burstCapacity));
if (log.isDebugEnabled()) {
log.debug("response: " + response);
}
return response;
});
}
catch (Exception e) {
/*
* We don't want a hard dependency on Redis to allow traffic. Make sure to set
* an alert so you know if this is happening too much. Stripe's observed
* failure rate is 0.01%.
*/
log.error("Error determining if user allowed from redis", e);
}
return Mono.just(new Response(true, getHeaders(-1L, replenishRate, burstCapacity)));
}
private static List<String> getKeys(String id) {
String prefix = "request_rate_limiter.{" + id;
String tokenKey = prefix + "}.tokens";
String timestampKey = prefix + "}.timestamp";
return Arrays.asList(tokenKey, timestampKey);
}
private HashMap<String, String> getHeaders(Long tokensLeft, Long replenish, Long burst) {
HashMap<String, String> headers = new HashMap<>();
headers.put(RedisRateLimiter.REMAINING_HEADER, tokensLeft.toString());
headers.put(RedisRateLimiter.REPLENISH_RATE_HEADER, replenish.toString());
headers.put(RedisRateLimiter.BURST_CAPACITY_HEADER, burst.toString());
return headers;
}
#Override
public Map getConfig() {
return rateLimiter.getConfig();
}
#Override
public Class getConfigClass() {
return rateLimiter.getConfigClass();
}
#Override
public Object newConfig() {
return rateLimiter.newConfig();
}
}
So, the route would look like this:
#Component
public class Routes {
#Autowired
ApiKeyRateLimiter rateLimiter;
#Autowired
ApiKeyResolver apiKeyResolver;
#Bean
public RouteLocator theRoutes(RouteLocatorBuilder b) {
return b.routes()
.route(p -> p
.path("/unlimited")
.uri("http://httpbin.org:80/anything?route=unlimited")
)
.route(p -> p
.path("/limited")
.filters(f ->
f.requestRateLimiter(r -> {
r.setKeyResolver(apiKeyResolver);
r.setRateLimiter(rateLimiter);
} )
)
.uri("http://httpbin.org:80/anything?route=limited")
)
.build();
}
}
Hope it saves a work day for somebody...
I have two applications communicating with each other using rabbit.
I need to send (from app1) an object to a listener (in app2) and after some process (on listener) it answer me with another object, now I am receiving this error:
ClassNotFound
I am using this config for rabbit in both applications:
#Configuration
public class RabbitConfiguration {
public final static String EXCHANGE_NAME = "paymentExchange";
public final static String EVENT_ROUTING_KEY = "eventRoute";
public final static String PAYEMNT_ROUTING_KEY = "paymentRoute";
public final static String QUEUE_EVENT = EXCHANGE_NAME + "." + "event";
public final static String QUEUE_PAYMENT = EXCHANGE_NAME + "." + "payment";
public final static String QUEUE_CAPTURE = EXCHANGE_NAME + "." + "capture";
#Bean
public List<Declarable> ds() {
return queues(QUEUE_EVENT, QUEUE_PAYMENT);
}
#Autowired
private ConnectionFactory rabbitConnectionFactory;
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(rabbitConnectionFactory);
}
#Bean
public DirectExchange exchange() {
return new DirectExchange(EXCHANGE_NAME);
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate r = new RabbitTemplate(rabbitConnectionFactory);
r.setExchange(EXCHANGE_NAME);
r.setChannelTransacted(false);
r.setConnectionFactory(rabbitConnectionFactory);
r.setMessageConverter(jsonMessageConverter());
return r;
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
private List<Declarable> queues(String... nomes) {
List<Declarable> result = new ArrayList<>();
for (int i = 0; i < nomes.length; i++) {
result.add(newQueue(nomes[i]));
if (nomes[i].equals(QUEUE_EVENT))
result.add(makeBindingToQueue(nomes[i], EVENT_ROUTING_KEY));
else
result.add(makeBindingToQueue(nomes[i], PAYEMNT_ROUTING_KEY));
}
return result;
}
private static Binding makeBindingToQueue(String queueName, String route) {
return new Binding(queueName, DestinationType.QUEUE, EXCHANGE_NAME, route, null);
}
private static Queue newQueue(String nome) {
return new Queue(nome);
}
}
I send the message using this:
String response = (String) rabbitTemplate.convertSendAndReceive(RabbitConfiguration.EXCHANGE_NAME,
RabbitConfiguration.PAYEMNT_ROUTING_KEY, domainEvent);
And await for a response using a cast to the object.
This communication is between two different applications using the same rabbit server.
How can I solve this?
I expected rabbit convert the message to a json in the send operation and the same in the reply, so I've created the object to correspond to a json of reply.
Show, please, the configuration for the listener. You should be sure that ListenerContainer there is supplied with the Jackson2JsonMessageConverter as well to carry __TypeId__ header back with the reply.
Also see Spring AMQP JSON sample for some help.
In my vaadin application, i need to use #Push, but since i added it, i can't read and write cookies because VaadinService.getSurrentResponse()returns null because of Push. I manager cookies using this class :
import javax.servlet.http.Cookie;
import com.vaadin.server.VaadinResponse;
import com.vaadin.server.VaadinService;
public class CookieManager {
private VaadinResponse response;
public CookieManager(VaadinResponse response){
this.response = response;
}
public Cookie getCookieByName(final String name) {
// Fetch all cookies from the request
Cookie[] cookies = VaadinService.getCurrentRequest().getCookies();
// Iterate to find cookie by its name
for (Cookie cookie : cookies) {
if (name.equals(cookie.getName())) {
return cookie;
}
}
return null;
}
public Cookie createCookie(final String name, final String value, final int maxAge) {
// Create a new cookie
final Cookie cookie = new Cookie(name, value);
cookie.setMaxAge(maxAge);
// Set the cookie path.
cookie.setPath(VaadinService.getCurrentRequest().getContextPath());
// Save cookie
addCookie(cookie);
return cookie;
}
private void addCookie(Cookie cookie){
response.addCookie(cookie);
}
public Cookie updateCookieValue(final String name, final String value) {
// Create a new cookie
Cookie cookie = getCookieByName(name);
cookie.setValue(value);
// Save cookie
addCookie(cookie);
return cookie;
}
public void destroyCookieByName(final String name) {
Cookie cookie = getCookieByName(name);
if (cookie != null) {
cookie.setValue(null);
// By setting the cookie maxAge to 0 it will deleted immediately
cookie.setMaxAge(0);
cookie.setPath(VaadinService.getCurrentRequest().getContextPath());
addCookie(cookie);
}
}
}
When i want to create a cookie (like at user's login), i get a nullPointerException because of the VaadinResponse being null.
So i tried to disable Push in constructor and re-enable it at the end of addCookie()method, but it disabled push for all of my application, even if i re-enable it just after the addCookiemethod.
I saw a ticket on vaadin's trac (http://dev.vaadin.com/ticket/11808) saying that will not be fixed, and someone suggested to create a regular AJAX query from server to create cookie, but i really don't know how to do.
How can i manage my cookies? i need to create AND get cookies, so javascript can't help me there, because i can't get javascript's return in vaadin, so i can't get a cookie.
Here is my solution how to store cookie when #Push is using.
First we create container to storage all instance of client UI. (
This container itself has a great potential)
public class UISession {
private List<WebAppUI> uis = new ArrayList<WebAppUI>();
public void addUI(WebAppUI webAppUI) {
uis.add(webAppUI);
}
public List<WebAppUI> getUIs() {
return uis;
}
public static UISession getInstance() {
try {
UI.getCurrent().getSession().lock();
return (UISession) UI.getCurrent().getSession().getAttribute("userUiSession");
} finally {
UI.getCurrent().getSession().unlock();
}
}
In UI.init() we add new instance to the session (e.g when user open new tab)
#Override
protected void init(VaadinRequest vaadinRequest) {
/** Set singleton uisesison for each browser*/
if(UISession.getInstance()==null){
UI.getCurrent().getSession().setAttribute("userUiSession",new UISession());
}
UISession.getInstance().addUI(this);
System.out.println("UI count fo current browser "+UISession.getInstance().getUIs().size());
...
}
Here is my helper cookie class:
class MyCookie{
private String value;
private String name;
private Date expired;
private String path="/";
public MyCookie(String name, String value) {
this.name=name;
this.value=value;
}
public void setMaxAge(int minute) {
Calendar c = Calendar.getInstance();
c.add(Calendar.MINUTE, minute);
expired=c.getTime();
}
public String getStringToCreateCookie(){
return "document.cookie=\""+getName()+"="+getValue()+"; expires="+expired.toString()+"; path="+path+"\"";
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
public Date getExpired() {
return expired;
}
public void setExpired(Date expired) {
this.expired = expired;
}
public String getPath() {
return path;
}
public void setPath(String path) {
this.path = path;
}
}
And on final when we need add new cookie, we just must find Ui that is active and call js function
public static void addCookie(String name, String value, int age){
MyCookie myCookie = new MyCookie(name, value);
myCookie.setMaxAge(age);
for(WebAppUI ui : UISession.getInstance().getUIs()){
if(ui.isAttached()){
ui.getPage().getJavaScript().execute(myCookie.getStringToCreateCookie());
return;
}
}
}
In my case i have access to storage cookie (when user made request). I just only have problem with add new cookie so this is my working solutions.
As mentioned in the ticket, you can use JavaScript to call client code and also request a cookie value back by that. E.g.
#Grapes([
#Grab('org.vaadin.spring:spring-boot-vaadin:0.0.3'),
#Grab('com.vaadin:vaadin-server:7.4.0.beta1'),
#Grab('com.vaadin:vaadin-client-compiled:7.4.0.beta1'),
#Grab('com.vaadin:vaadin-themes:7.4.0.beta1'),
])
import com.vaadin.ui.*
#org.vaadin.spring.VaadinUI
#groovy.transform.CompileStatic
class MyUI extends UI {
protected void init(com.vaadin.server.VaadinRequest request) {
final resultLabel = new Label()
// provide a callback for the client to tell the cookies
JavaScript.current.addFunction("tellCookie", { elemental.json.JsonArray arguments ->
resultLabel.value = arguments?.get(0)?.asString()
} as JavaScriptFunction)
setContent(new VerticalLayout().with{
addComponent(new Button("Set Cookie", {
// just simply set the cookies via JS (attn: quoting etc)
JavaScript.current.execute("document.cookie='mycookie=${System.currentTimeMillis()}'")
} as Button.ClickListener))
addComponent(new Button("Get Cookie", {
// tell the client to tell the server the cookies
JavaScript.current.execute("this.tellCookie(document.cookie)")
} as Button.ClickListener))
addComponent(resultLabel)
return it
})
}
}
This is a running example (e.g. spring run vaadin.groovy) for testing. See the comments for the important parts.
The Viritin add-on contains a helper class called BrowserCookie. It works in pretty much the way suggested by cfrick, but just hides all the cookie handling complexity into a helper class. It don't contain built in "max age" handling yet, but that could be easily added as a workaround you can manually "encode" the age into cookie value.
BTW. Don't know what you are doing, but if you happen to be using TouchKit add-on, it has a helper for html5 local storage. It has rather wide browsers support already and is in many ways better way to store e.g. settings than cookies.