I have the following Rhino ETL process which fails silently. The process executes just fine however no round-trips are ever made to the DB, no exceptions are thrown and no data is ever inserted.
public class UpdateLeadSources : EtlProcess
{
protected override void Initialize()
{
Register(new ConventionInputCommandOperation("MoxyVote")
{
Command = "Select * from dbo.LeadSources"
});
Register(new ConventionOutputCommandOperation("MoxyDataWarehouse")
{
Command = "Insert into dbo.LeadSources (LeadSourceID, LeadSourceCategoryID, LeadSourceCode, LeadSourceFriendlyName, Description, IsActive, Password, TopEntityID, TopEntityTypeID, CampaignID) Values(#LeadSourceID, #LeadSourceCategoryID, #LeadSourceCode, #LeadSourceFriendlyName, #Description, #IsActive, #Password, #TopEntityID, #TopEntityTypeID, #CampaignID)"
});
}
}
rhino etl will catch all exceptions, log them and add them to a collection of errors you can access the collection from the EtlProcess class.
https://github.com/hibernating-rhinos/rhino-etl/blob/master/Rhino.Etl.Core/EtlProcess.cs#L161 line 161 GetAllErrors()
Related
I try to run a test with MockWebServer.
I would like to make a UI test with mocked response, so I could test for valid\invalid UI changes like logging in, or showing error in a login API.
However, each and every time I ran the code I got the following exception:
java.lang.IllegalStateException: start() already called
Code:
#RunWith(AndroidJUnit4.class)
public class UITestPlayground {
String testUrl = "http://testurl.com/";
MockWebServer server = new MockWebServer();
#Rule
public IntentsTestRule<LoginActivity> mIntentsRule = new IntentsTestRule<>(LoginActivity.class);
#Before
public void beforeHelper() throws IOException {
TestHelper.removeUserAndTokenIfAny(mIntentsRule.getActivity());
URLS.baseUrl = testUrl;
server.url(URLS.baseUrl);
//try to shutting down the server JUT IN CASE...
server.shutdown();
server.start();
}
#After
public void afterHelper() throws IOException {
server.shutdown();
}
#Test
public void invalidLoginDueNotValidJSONResponse() {
server.enqueue(new MockResponse().setBody("Something not valid JSON response"));
String emailToBeTyped = "tester#tester.com";
String passToBeTyped = "passtest";
ViewActions.closeSoftKeyboard();
// Type text and then press the button.
onView(withId(R.id.login_email_edit)).perform(typeText(emailToBeTyped));
ViewActions.closeSoftKeyboard();
onView(withId(R.id.login_pass_edit)).perform(typeText(passToBeTyped));
ViewActions.closeSoftKeyboard();
onView(withId(R.id.log_in_btn)).perform(click());
//TODO: check on valid error message appearing
}
}
What am I doing wrong? The .start() only called once, I even .shutdown() just in
case... I don't understand how could it called more than once.
Thanks in advance.
In the original example at github I have found that the order is reversed.
You actually start the server, THEN sets it's url.
And not setting the url then starting the server.
Interesting.
There are some database operations I need to execute before the end of the final attempt of my Hangfire background job (I need to delete the database record related to the job)
My current job is set with the following attribute:
[AutomaticRetry(Attempts = 5, OnAttemptsExceeded = AttemptsExceededAction.Delete)]
With that in mind, I need to determine what the current attempt number is, but am struggling to find any documentation in that regard from a Google search or Hangfire.io documentation.
Simply add PerformContext to your job method; you'll also be able to access your JobId from this object. For attempt number, this still relies on magic strings, but it's a little less flaky than the current/only answer:
public void SendEmail(PerformContext context, string emailAddress)
{
string jobId = context.BackgroundJob.Id;
int retryCount = context.GetJobParameter<int>("RetryCount");
// send an email
}
(NB! This is a solution to the OP's problem. It does not answer the question "How to get the current attempt number". If that is what you want, see the accepted answer for instance)
Use a job filter and the OnStateApplied callback:
public class CleanupAfterFailureFilter : JobFilterAttribute, IServerFilter, IApplyStateFilter
{
public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
try
{
var failedState = context.NewState as FailedState;
if (failedState != null)
{
// Job has finally failed (retry attempts exceeded)
// *** DO YOUR CLEANUP HERE ***
}
}
catch (Exception)
{
// Unhandled exceptions can cause an endless loop.
// Therefore, catch and ignore them all.
// See notes below.
}
}
public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
// Must be implemented, but can be empty.
}
}
Add the filter directly to the job function:
[CleanupAfterFailureFilter]
public static void MyJob()
or add it globally:
GlobalJobFilters.Filters.Add(new CleanupAfterFailureFilter ());
or like this:
var options = new BackgroundJobServerOptions
{
FilterProvider = new JobFilterCollection { new CleanupAfterFailureFilter () };
};
app.UseHangfireServer(options, storage);
Or see http://docs.hangfire.io/en/latest/extensibility/using-job-filters.html for more information about job filters.
NOTE: This is based on the accepted answer: https://stackoverflow.com/a/38387512/2279059
The difference is that OnStateApplied is used instead of OnStateElection, so the filter callback is invoked only after the maximum number of retries. A downside to this method is that the state transition to "failed" cannot be interrupted, but this is not needed in this case and in most scenarios where you just want to do some cleanup after a job has failed.
NOTE: Empty catch handlers are bad, because they can hide bugs and make them hard to debug in production. It is necessary here, so the callback doesn't get called repeatedly forever. You may want to log exceptions for debugging purposes. It is also advisable to reduce the risk of exceptions in a job filter. One possibility is, instead of doing the cleanup work in-place, to schedule a new background job which runs if the original job failed. Be careful to not apply the filter CleanupAfterFailureFilter to it, though. Don't register it globally, or add some extra logic to it...
You can use OnPerforming or OnPerformed method of IServerFilter if you want to check the attempts or if you want you can just wait on OnStateElection of IElectStateFilter. I don't know exactly what requirement you have so it's up to you. Here's the code you want :)
public class JobStateFilter : JobFilterAttribute, IElectStateFilter, IServerFilter
{
public void OnStateElection(ElectStateContext context)
{
// all failed job after retry attempts comes here
var failedState = context.CandidateState as FailedState;
if (failedState == null) return;
}
public void OnPerforming(PerformingContext filterContext)
{
// do nothing
}
public void OnPerformed(PerformedContext filterContext)
{
// you have an option to move all code here on OnPerforming if you want.
var api = JobStorage.Current.GetMonitoringApi();
var job = api.JobDetails(filterContext.BackgroundJob.Id);
foreach(var history in job.History)
{
// check reason property and you will find a string with
// Retry attempt 3 of 3: The method or operation is not implemented.
}
}
}
How to add your filter
GlobalJobFilters.Filters.Add(new JobStateFilter());
----- or
var options = new BackgroundJobServerOptions
{
FilterProvider = new JobFilterCollection { new JobStateFilter() };
};
app.UseHangfireServer(options, storage);
Sample output :
I have an application which basically calls multiple webservices, stores the data received from those webservices and renders it to the user. I have an async task that call all the webservices and it looks something like this:
List<Promise> t = new ArrayList<Promise>()
def TIMEOUT_TIME = 6
t[0] = task {
def responseFrom0 = webserviceRequestService.getResponse(0)
if(responseFrom0){
return responseFrom0
}
}
t[1] = task {
def responseFrom1 = webserviceRequestService.getResponse(1)
if(responseFrom1){
return responseFrom1
}
}
The action getResponse looks something like this:
List<ResponseResult> result = new ArrayList<TravelQuote>()
try {
wsClient = prepareRequestMap()
wsResponse = externalWebservice.getQuotes(wsClient)
wsResponse.responseList.each {
ResponseResult responseResult = new ResponseResult()
//populate properties of ResponseResult
responseResult.save(failOnError:true, flush:true)
result.add(responseResult)
}
} catch(Exception e){
log.error e.message
e.printStackTrace()
}
return result
And at the end, I collect all the responses from all webservices like this:
result.each {
if(it){
try{
it=it.merge()
}catch (Exception e){
log.error("Error occured while merging responses... : ${e.message}")
}
}
}
Now, the issue here is I get this exception from the last block of code
not-null property references a null or transient value: ResponseResult.dateCreated; nested exception is org.hibernate.PropertyValueException: not-null property references a null or transient value: ResponseResult.dateCreated
The dateCreated comes from this class which I have implemented on all of my domain classes.
abstract class AbstractPersistentObject implements Serializable{
static final long serialVersionUID = 1L;
Date dateCreated
Date lastUpdated
}
The weird thing about this issue is, this happens only on production environment, no matter what I do, I cannot replicate this in any other environments. And also, once this happens, the server just throws that particular issue every time that code is run until the server is restarted. After a restart, this code works fine.
I am running out of ideas, anyone has any ideas?
The answer seems to be in the exception thrown: the external web service returns record(s) without value for dateCreated field.
In my experience data from production environments almost always contains missing or improperly formatted values. You should account for that by changing the definition of ResponseResult.dateCreated to allow null values and handle this scenario in your code.
I am using Edge.js so that I can call Node.js from C#. According to the documentation in the link I would do that similar to the following:
[HttpPost]
public ActionResult Input(InputModel obj)
{
validateInput(obj);
return View();
}
private async void validateInput(object obj)
{
var func = Edge.Func(#"
return function (data, callback){
var username = data.username,
email = data.email;
callback(null, username);
}
");
ViewBag.Msg = (string)await func(obj);
}
However, I get the following run time error:
Additional information: An asynchronous operation cannot be started at this time.
Asynchronous operations may only be started within an asynchronous handler or module or during
certain events in the Page lifecycle. If this exception occurred while executing a Page, ensure that the
Page is marked <%# Page Async="true" %>. This exception may also indicate an attempt to call an
"async void" method, which is generally unsupported within ASP.NET request processing. Instead,
the asynchronous method should return a Task, and the caller should await it.
My question is two-fold:
1.How do I make the page, async=true. I know how to do this for a web forms project but not a MVC project.
2.Is there a better way to do what I am trying to do? A red flag will probably go up when you see that I am returning void however this is do to the fact that Edge.js is being used. Even so, I have tried returning a Task and then task.Wait() in the calling method but the task never finishes.
After trying some different things, the following solution worked for me.
Even though I answered my own question, and it seems trivial, I am not removing this question as there are not a lot of knowledge on the web about Edge.js.
[HttpPost]
public async Task<ActionResult> Input(InputModel obj)
{
ViewBag.Msg = await validateInput(obj);
return View();
}
private async Task<string> validateInput(object obj)
{
var func = Edge.Func(#"
return function (data, callback){
var username = data.username,
email = data.email;
callback(null, username);
}
");
return (string)await func(obj);
}
I have junit testscript that creates different and unique ID. So when it finds an existing ID or a wrong Id I want the test script report via ANT to show that it is failed for following record but passed for the rest of the records that are correct.
#Test
public void testCreateTrade() throws Exception
driver.findElement(By.id("VIN")).clear();
driver.findElement(By.id("VIN")).sendKeys(vVin);
String str = driver.getCurrentUrl();
if(str.contains("step1")) // for existing ID
{
driver.findElement(By.cssSelector("body > div.bootbox.modal.in > div.modal-footer > a.btn.null")).click();
break;
}
driver.findElement(By.id("mileage")).sendKeys(vMileage);
driver.findElement(By.id("odometerType")).sendKeys(vKm);
driver.findElement(By.id("passengers")).sendKeys(vPassengers);
driver.findElement(By.id("exteriorColor")).sendKeys(vExterior);
driver.findElement(By.id("interiorColor")).sendKeys(vInterior);
driver.findElement(By.id("hasAccident")).sendKeys(vAccident);
driver.findElement(By.id("dealerSalesPerson")).sendKeys(vSalesPerson);
driver.findElement(By.id("step3btn")).click();
Thread.sleep(1000);
String str3 = driver.getCurrentUrl();
if(str3.contains("step2")) // Loop for wrong ID
{
driver.findElement(By.linkText("Create")).click();
driver.findElement(By.xpath("html/body/div[7]/div[2]/a[1]")).click();
//System.out.println("Is a wrong Vin"+vVin);
break;
}
driver.findElement(By.id("step4btn")).click();
driver.findElement(By.id("windshieldCondition")).sendKeys(vWindshield);
driver.findElement(By.id("tireCondition")).sendKeys(vTire);
driver.findElement(By.id("accidentBrand3")).sendKeys(vAcBrand);
driver.findElement(By.id("confirmedParked")).click();
If you want a single test case to continue running after it "fails" and then report its exceptions at the end, use ErrorCollector.
#RunWith(JUnit4.class) public class YourTestClass {
#Rule public ErrorCollector errorCollector = new ErrorCollector();
#Test public void yourTest() {
// ... (your setup)
for (Record record : expectedRecords) {
if (dataSource.hasRecord(record.getId())) {
Record fetchedRecord = dataSource.getRecord(record.getId());
errorCollector.checkThat(record, matchesRecordValuesOf(record));
} else {
errorCollector.addError(new IllegalStateException(""));
}
}
}
}
Note, however, that it's always preferable to test exactly one thing per unit test. Here it may make sense, but don't overuse ErrorCollector where refactoring and splitting the test makes more sense.