public Mono<Response> createSomething(PostRequest request, Option option, SiteId siteId) {
Predicate<DeliveryPromiseResponse> isSplitShipment = pudoDPEResponse -> pudoDPEResponse.getShipments().size() > 1;
return Mono.just(request)
.doFirst(() -> log.debug("Processing request for siteId :", request.getSiteId()))
.flatMap(shipRequest -> converter.apply(shipRequest, siteId))
.filter(dpeResponse -> {
if (isSplitShipment.test(dpeResponse)) {
log.info("Received Shipment response from DPE for siteId :", request.getSiteId());
return false;
}
return true;
})};
Although there might be another ways to achieve this behavior, I would personally refactor in two steps:
1. Extract filtering to separate step:
.filter(dpeResponse -> !isSplitShipment.test(dpeResponse))
To improve readibility, I would personally extract it to method called something like isNotSplitShipment.
2. Extract side effect (logging) to doOnSuccess method:
.doOnSuccess(dpeResponse -> {
if (dpeResponse == null)
log.info("Received Shipment response from DPE for siteId :", request.getSiteId());
})
Of course, this one could be extracted to a separate method, as well.
All together, will look, as follows:
return Mono.just(request)
.doFirst(() -> log.debug("Processing request for siteId :", request.getSiteId()))
.flatMap(shipRequest -> converter.apply(shipRequest, siteId))
.filter(dpeResponse -> !isSplitShipment.test(dpeResponse))
.doOnSuccess(dpeResponse -> {
if (dpeResponse == null)
log.info("Received Shipment response from DPE for siteId :", request.getSiteId());
});
This way, we avoid mixing filtering with logging. Moreover, return false/true are not required anymore.
Nevertheless, it's worth mentioning here, that the doOnSuccess should be used cautiously. It should not invoke another Mono/Fluxes as it will not propagate errors to the main sequence.
Related
I have a method that takes a list of items, makes a web request and then returns items that failed processing (some items may have been processed correctly, some items might have failed and only failed ones are returned or an empty list):
// returns list of of items that failed processing or empty list.
private List<String> sendsItems(List<String> items);
I want to use Mono and retry sending only failed items for up to N times. Using a blocking code, it would look like this:
public void sendWithRetries() {
List<String> items = List.of("one", "two", "three");
int retryNo = 0;
while (!items.isEmpty() && retryNo < 3) {
items = sendsItems(items);
retryNo++;
}
}
I have really hard time translating that to code using reactor. I've been thinking about using Mono.repeat or Mono.retry family of functions but I do not see any nice way of passing failed items back to sendItems except doing something ugly like:
var items = new AtomicReference<>(List.of("one", "two", "three"));
Mono.fromCallable(() -> {
List<String> failedItems = sendsItems(items.get());
items.set(failedItems);
if (!failedItems.isEmpty()) {
throw new RuntimeException("retry me");
}
return Collections.emptyList();
})
.retry(3)
Is there a better way of doing that?
note: the real use case I need it for is kinesis put records api call. It returns info about which records failed to be processed and I want to retry sending only them.
You can use expand operator to maintain state:
Mono.fromCallable(() -> sendsItems(new Attempt(List.of("one", "two", "three"), 0)))
.expand(attempt -> {
if (attempt.getItems().isEmpty() || attempt.getRetry() > 3) {
return Mono.empty();
} else {
return Mono.fromCallable(() -> sendsItems(attempt));
}
});
private Attempt sendsItems(Attempt previousAttempt) {
System.out.println(previousAttempt);
// implement actual sending logic here, below is just some dummy
List<String> failedItems = previousAttempt.getItems().subList(0, previousAttempt.getItems().size() - 1);
return new Attempt(failedItems, previousAttempt.retry + 1);
}
#Value
static class Attempt {
List<String> items;
int retry;
}
In a piece of code using language-ext library, I can perform an async action only when the Option<> intermediate result is actually filled:
async Task<Option<MyEntity>> FindEntityAsync(string entityId)
{
Option<MyEntity> entityOpt = await GetEntityAsync(entityId);
if (entityOpt.IsSome)
{
await DoSomethingAsync(entityOpt.First());
}
return entityOpt;
}
// Task<Option<MyEntity>> GetEntityAsync(string entityId) { ... }
// DoSomethingAsync could either be:
// Task DoSomethingAsync(MyEntity entity) { ... }
// or:
// Task<Unit> DoSomethingAsync(MyEntity entity)
I'm looking for a more idiomatic way (for such library) to achieve the same.
I tried the following but it does not work:
// look ma! No async/await here
Task<Option<MyEntity>> FindEntityAsync(string entityId)
{
Task<Option<MyEntity>> result =
from entity in GetEntityAsync(entityId)
from _ in DoSomethingAsync(entity).Map(Some)
select entity;
return result;
}
I experienced some LanguageExt.ValueIsNoneException when the Option<> is None.
Ideally I'd like to user an IterXxx type of operator in order to traverse the wrapped Option<MyEntity> only when there is something:
Task<Option<MyEntity>> FindEntityAsync(string entityId)
{
Task<Option<MyEntity>> result = GetEntityAsync(entityId);
result.IterXxxx(async entity => await DoSomethingAsync(entity));
return result;
}
but I cannot find any suitable signature working with an async action. Any hint?
There is special type OptionAsync<T> for combining two monads - Option and Async.
For more info please refer to github - https://github.com/louthy/language-ext/issues/206
I have implemented a dummy reactive repository but I am struggling with the update method:
#Override
public Mono<User> updateUser(int id, Mono<User> updateMono) {
return //todo with getUser
}
#Override
public Mono<User> getUser(int id) {
return Mono.justOrEmpty(this.users.get(id));
}
From one hand I have incoming publisher Mono<User> updateMono, from the other hand I have another publisher during Mono.justOrEmpty(this.users.get(id)).
How to combine it together, make update, and give back just one publisher?
The only thing come to my mind is:
#Override
public Mono<User> updateUser(int id, Mono<User> updateMono) {
return getUser(id).doOnNext(user -> {
updateMono.subscribe(update -> {
users.put(id, new User(id, update.getName(), update.getAge()));
System.out.format("Updated user with id %d to %s%n", id, update);
});
});
}
Is it correct?
See the reference guide on finding the right operator
Notably, for Mono you have and, when, then (note this last one will become flatMap in 3.1.0, and flatmap will become flatMapMany)
doOnNext is more for side operations like logging or stats gathering. Subscribe inside subscribe is another bad form; generally you want flatMap or similar instead.
I have played Spring 5 Reactive Streams features in these days, and have written down some sample codes(not public via blog or twitter yet, I still need more practice on Reactor).
I have encountered the same problems and finally used a Mono.zip to update the existing item in MongoDB.
https://github.com/hantsy/spring-reactive-sample/blob/master/boot-routes/src/main/java/com/example/demo/DemoApplication.java
public Mono<ServerResponse> update(ServerRequest req) {
return Mono
.zip(
(data) -> {
Post p = (Post) data[0];
Post p2 = (Post) data[1];
p.setTitle(p2.getTitle());
p.setContent(p2.getContent());
return p;
},
this.posts.findById(req.pathVariable("id")),
req.bodyToMono(Post.class)
)
.cast(Post.class)
.flatMap(post -> this.posts.save(post))
.flatMap(post -> ServerResponse.noContent().build());
}
Update: Another working version written in Kotlin.
fun update(req: ServerRequest): Mono<ServerResponse> {
return this.posts.findById(req.pathVariable("id"))
.and(req.bodyToMono(Post::class.java))
.map { it.t1.copy(title = it.t2.title, content = it.t2.content) }
.flatMap { this.posts.save(it) }
.flatMap { noContent().build() }
}
There are some database operations I need to execute before the end of the final attempt of my Hangfire background job (I need to delete the database record related to the job)
My current job is set with the following attribute:
[AutomaticRetry(Attempts = 5, OnAttemptsExceeded = AttemptsExceededAction.Delete)]
With that in mind, I need to determine what the current attempt number is, but am struggling to find any documentation in that regard from a Google search or Hangfire.io documentation.
Simply add PerformContext to your job method; you'll also be able to access your JobId from this object. For attempt number, this still relies on magic strings, but it's a little less flaky than the current/only answer:
public void SendEmail(PerformContext context, string emailAddress)
{
string jobId = context.BackgroundJob.Id;
int retryCount = context.GetJobParameter<int>("RetryCount");
// send an email
}
(NB! This is a solution to the OP's problem. It does not answer the question "How to get the current attempt number". If that is what you want, see the accepted answer for instance)
Use a job filter and the OnStateApplied callback:
public class CleanupAfterFailureFilter : JobFilterAttribute, IServerFilter, IApplyStateFilter
{
public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
try
{
var failedState = context.NewState as FailedState;
if (failedState != null)
{
// Job has finally failed (retry attempts exceeded)
// *** DO YOUR CLEANUP HERE ***
}
}
catch (Exception)
{
// Unhandled exceptions can cause an endless loop.
// Therefore, catch and ignore them all.
// See notes below.
}
}
public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
// Must be implemented, but can be empty.
}
}
Add the filter directly to the job function:
[CleanupAfterFailureFilter]
public static void MyJob()
or add it globally:
GlobalJobFilters.Filters.Add(new CleanupAfterFailureFilter ());
or like this:
var options = new BackgroundJobServerOptions
{
FilterProvider = new JobFilterCollection { new CleanupAfterFailureFilter () };
};
app.UseHangfireServer(options, storage);
Or see http://docs.hangfire.io/en/latest/extensibility/using-job-filters.html for more information about job filters.
NOTE: This is based on the accepted answer: https://stackoverflow.com/a/38387512/2279059
The difference is that OnStateApplied is used instead of OnStateElection, so the filter callback is invoked only after the maximum number of retries. A downside to this method is that the state transition to "failed" cannot be interrupted, but this is not needed in this case and in most scenarios where you just want to do some cleanup after a job has failed.
NOTE: Empty catch handlers are bad, because they can hide bugs and make them hard to debug in production. It is necessary here, so the callback doesn't get called repeatedly forever. You may want to log exceptions for debugging purposes. It is also advisable to reduce the risk of exceptions in a job filter. One possibility is, instead of doing the cleanup work in-place, to schedule a new background job which runs if the original job failed. Be careful to not apply the filter CleanupAfterFailureFilter to it, though. Don't register it globally, or add some extra logic to it...
You can use OnPerforming or OnPerformed method of IServerFilter if you want to check the attempts or if you want you can just wait on OnStateElection of IElectStateFilter. I don't know exactly what requirement you have so it's up to you. Here's the code you want :)
public class JobStateFilter : JobFilterAttribute, IElectStateFilter, IServerFilter
{
public void OnStateElection(ElectStateContext context)
{
// all failed job after retry attempts comes here
var failedState = context.CandidateState as FailedState;
if (failedState == null) return;
}
public void OnPerforming(PerformingContext filterContext)
{
// do nothing
}
public void OnPerformed(PerformedContext filterContext)
{
// you have an option to move all code here on OnPerforming if you want.
var api = JobStorage.Current.GetMonitoringApi();
var job = api.JobDetails(filterContext.BackgroundJob.Id);
foreach(var history in job.History)
{
// check reason property and you will find a string with
// Retry attempt 3 of 3: The method or operation is not implemented.
}
}
}
How to add your filter
GlobalJobFilters.Filters.Add(new JobStateFilter());
----- or
var options = new BackgroundJobServerOptions
{
FilterProvider = new JobFilterCollection { new JobStateFilter() };
};
app.UseHangfireServer(options, storage);
Sample output :
So I have the following action which I am trying to add output caching to:
[OutputCache(CacheProfile = OutputCacheProfileNames.Hours24)]
public ActionResult ContactUs()
{
ContactUsModel model = _modelBuilder.BuildContactUsModel();
if (Request.IsAjaxRequest())
{
return Json(StringFromPartial(partialTemplate, model), JsonRequestBehavior.AllowGet);
}
else
{
return View(model);
}
}
But this seem to cache the first view that is requested - ie either the json OR the normal view.
Is there a way to get the output caching to work for both views, without having to split them out of the same action?
You beat me to the punch in answering your own question, but I thought this code may still be helpful. Since varying by user is such a common scenario, you should probably account for being able to do that and your AJAX vary. This code will allow you vary on any number of custom parameters, by appending to a single string to vary on.
public override string GetVaryByCustomString(System.Web.HttpContext context, string custom)
{
var args = custom.ToLower().Split(';');
var sb = new StringBuilder();
foreach (var arg in args)
{
switch (arg)
{
case "user":
sb.Append(User.Identity.Name);
break;
case "ajax":
if (context.Request.Headers["X-Requested-With"] != null)
{
// "XMLHttpRequest" will be appended if it's an AJAX request
sb.Append(context.Request.Headers["X-Requested-With"]);
}
break;
default:
continue;
}
}
return sb.ToString();
}
Then, you would just do something like the following if you need to vary by multiple custom params.
[OutputCache(CacheProfile = OutputCacheProfileNames.Hours24, VaryByCustom = "User;Ajax")]
Then, if you ever need additional custom vary params, you just keep adding case statements to cover those scenarios.
Thanks to the comments by REDEVI_ for pointing me in the right direction, I have been able to solve this.
I changed my output caching to:
[OutputCache(CacheProfile = OutputCacheProfileNames.Hours24, VaryByCustom = "IsAjax")]
And then in my global.asax file, I added the following override:
public override string GetVaryByCustomString(HttpContext context, string custom)
{
if (context != null)
{
switch (custom)
{
case "IsAjax":
return new HttpRequestWrapper(context.Request).IsAjaxRequest() ? "IsAjax" : "IsNotAjax";
}
}
return base.GetVaryByCustomString(context, custom);
}