I have a Grails 4 application providing a REST API. One of the endpoints sometimes fail with the following exception:
io.micronaut.http.client.exceptions.ReadTimeoutException: Read Timeout
at io.micronaut.http.client.exceptions.ReadTimeoutException.<clinit>(ReadTimeoutException.java:26)
at io.micronaut.http.client.DefaultHttpClient$10.exceptionCaught(DefaultHttpClient.java:1917)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:276)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:268)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:426)
at io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:92)
at io.netty.channel.CombinedChannelDuplexHandler$1.fireExceptionCaught(CombinedChannelDuplexHandler.java:147)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.CombinedChannelDuplexHandler.exceptionCaught(CombinedChannelDuplexHandler.java:233)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:276)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:268)
at io.netty.handler.timeout.ReadTimeoutHandler.readTimedOut(ReadTimeoutHandler.java:98)
at io.netty.handler.timeout.ReadTimeoutHandler.channelIdle(ReadTimeoutHandler.java:90)
at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:505)
at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:477)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
The endpoint uses micronaut http client to call other systems. The remote system takes a very long time to respond, causing the ReadTimeOutException.
Here is the code calling the remote Service:
class RemoteTaskService implements GrailsConfigurationAware {
String taskStepperUrl
// initializes fields from configuration
void setConfiguration(Config config) {
taskStepperUrl = config.getProperty('services.stepper')
}
private BlockingHttpClient getTaskClient() {
HttpClient.create(taskStepperUrl.toURL()).toBlocking()
}
List<Map> loadTasksByProject(long projectId) {
try {
retrieveRemoteList("/api/tasks?projectId=${projectId}")
} catch(HttpClientResponseException e) {
log.error("Loading tasks of project failed with status: ${e.status.code}: ${e.message}")
throw new NotFoundException("No tasks found for project ${projectId}")
}
}
private List<Map> retrieveRemoteList(String path) {
HttpRequest request = HttpRequest.GET(path)
HttpResponse<List> response = taskClient.exchange(request, List) as HttpResponse<List>
response.body()
}
}
I've tried resolving it using the following configuration in my application.yml:
micronaut:
server:
read-timeout: 30
and
micronaut.http.client.read-timeout: 30
...with no success. Despite my configuration, the timeout still occurs around 10s after calling the endpoint.
How can I change the read timeout duration for the http rest client?
micronaut.http.client.read-timeout takes a duration, so you should add a measuring unit to the value, like 30s, 30m or 30h.
It seems that the configuration values are not injected in the manually created http clients.
A solution is to configure the HttpClient at creation, setting the readTimeout duration:
private BlockingHttpClient getTaskClient() {
HttpClientConfiguration configuration = new DefaultHttpClientConfiguration()
configuration.readTimeout = Duration.ofSeconds(30)
new DefaultHttpClient(taskStepperUrl.toURL(), configuration).toBlocking()
}
In my case I was streaming a file from a client as
#Get(value = "${service-path}", processes = APPLICATION_OCTET_STREAM)
Flowable<byte[]> fullImportStream();
so when I got this my first impulse was to increase the read-timeout value. Though, for streaming scenarios the property that applies is read-idle-timeout as stated in the docs https://docs.micronaut.io/latest/guide/configurationreference.html#io.micronaut.http.client.DefaultHttpClientConfiguration
Related
I am using HttpClient to make a couple of requests instead of httpwebrequest. everything works fine specially when I make the first request.However when I make the second requests always falls and I end up with getting the A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. I did follow all the suggested answers from other posts but none of them has helped and also compared to httpwebrequest all the requests works fine even if I made more than one request.
private async Task <string> getSomeData(string jobUrl)
{
Uri myUri = new Uri(jobUrl, UriKind.Absolute);
//GET Method
HttpResponseMessage response = await client.GetAsync(myUri);
if (response.IsSuccessStatusCode)
{
using (HttpContent content = response.Content)
{
String result = await response.Content.ReadAsStringAsync();
if (result != null)
{
return result;
}
}
}
return "";
}
I did also set the deafult proxy to false:
<system.net>
<defaultProxy enabled="false">
</defaultProxy>
</system.net>
EDIT: here https://github.com/wujek-srujek/reactor-retry-test is a repository with all the code.
I have the following Spring WebClient code to POST to a remote server (Kotlin code without imports for brevity):
private val logger = KotlinLogging.logger {}
#Component
class Client(private val webClient: WebClient) {
companion object {
const val maxRetries = 2L
val firstBackOff = Duration.ofSeconds(5L)
val maxBackOff = Duration.ofSeconds(20L)
}
fun send(uri: URI, data: Data): Mono<Void> {
return webClient
.post()
.uri(uri)
.contentType(MediaType.APPLICATION_JSON)
.bodyValue(data)
.retrieve()
.toBodilessEntity()
.doOnSubscribe {
logger.info { "Calling backend, uri: $uri" }
}
.retryExponentialBackoff(maxRetries, firstBackOff, maxBackOff, jitter = false) {
logger.debug { "Call to $uri failed, will retry (#${it.iteration()} of max $maxRetries)" }
}
.doOnError {
logger.error { "Call to $uri with $maxRetries retries failed with $it" }
}
.doOnSuccess {
logger.info { "Call to $uri succeeded" }
}
.then()
}
}
(It returns an empty Mono as we don't expect an answer, nor do we care about it.)
I would like to test 2 cases, and one of them is giving me headaches, namely the one in which I want to test that all the retries have been fired. We are using MockWebServer (https://github.com/square/okhttp/tree/master/mockwebserver) and the StepVerifier from reactor-test. (The test for success is easy and doesn't need any virtual time scheduler magic, and works just fine.) Here is the code for the failing one:
#JsonTest
#ContextConfiguration(classes = [Client::class, ClientConfiguration::class])
class ClientITest #Autowired constructor(
private val client: Client
) {
lateinit var server: MockWebServer
#BeforeEach
fun `init mock server`() {
server = MockWebServer()
server.start()
}
#AfterEach
fun `shutdown server`() {
server.shutdown()
}
#Test
fun `server call is retried and eventually fails`() {
val data = Data()
val uri = server.url("/server").uri()
val responseStatus = HttpStatus.INTERNAL_SERVER_ERROR
repeat((0..Client.maxRetries).count()) {
server.enqueue(MockResponse().setResponseCode(responseStatus.value()))
}
StepVerifier.withVirtualTime { client.send(uri, data) }
.expectSubscription()
.thenAwait(Duration.ofSeconds(10)) // wait for the first retry
.expectNextCount(0)
.thenAwait(Duration.ofSeconds(20)) // wait for the second retry
.expectNextCount(0)
.expectErrorMatches {
val cause = it.cause
it is RetryExhaustedException &&
cause is WebClientResponseException &&
cause.statusCode == responseStatus
}
.verify()
// assertions
}
}
I am using withVirtualTime because I don't want the test to take nearly seconds.
The problem is that the test blocks indefinitely. Here is the (simplified) log output:
okhttp3.mockwebserver.MockWebServer : MockWebServer[51058] starting to accept connections
Calling backend, uri: http://localhost:51058/server
MockWebServer[51058] received request: POST /server HTTP/1.1 and responded: HTTP/1.1 500 Server Error
Call to http://localhost:51058/server failed, will retry (#1 of max 2)
Calling backend, uri: http://localhost:51058/server
MockWebServer[51058] received request: POST /server HTTP/1.1 and responded: HTTP/1.1 500 Server Error
Call to http://localhost:51058/server failed, will retry (#2 of max 2)
As you can see, the first retry works, but the second one blocks. I don't know how to write the test so that it doesn't happen. To make matters worse, the client will actually use jitter, which will make the timing hard to anticipate.
The following test using StepVerifier but without WebClient works fine, even with more retries:
#Test
fun test() {
StepVerifier.withVirtualTime {
Mono
.error<RuntimeException>(RuntimeException())
.retryExponentialBackoff(5,
Duration.ofSeconds(5),
Duration.ofMinutes(2),
jitter = true) {
println("Retrying")
}
.then()
}
.expectSubscription()
.thenAwait(Duration.ofDays(1)) // doesn't matter
.expectNextCount(0)
.expectError()
.verify()
}
Could anybody help me fix the test, and ideally, explain what is wrong?
This is a limitation of virtual time and the way the clock is manipulated in StepVerifier. The thenAwait methods are not synchronized with the underlying scheduling (that happens for example as part of the retryBackoff operation). This means that the operator submits retry tasks at a point where the clock has already been advanced by one day. So the second retry is scheduled for + 1 day and 10 seconds, since the clock is at +1 day. After that, the clock is never advanced so the additional request is never made to MockWebServer.
Your case is made even more complicated in the sense that there is an additional component involved, the MockWebServer, that still works "in real time".
Though advancing the virtual clock is a very quick operation, the response from the MockWebServer still goes through a socket and thus has some amount of latency to the retry scheduling, which makes things more complicated from the test writing perspective.
One possible solution to explore would be to externalize the creation of the VirtualTimeScheduler and tie advanceTimeBy calls to the mockServer.takeRequest(), in a parallel thread.
I am trying to test ActiveMQ connection and return a value. it crashes on line:
httpResponse = client.execute(theHttpGet);
It is not my code I am trying to debug it. Can anyone help me to understand why the code is using HttpGet?
public ActivemqBrokerInfo(String serverAddress, int port, String apiUrl, int timeout) {
// Default Activemq location
this.serverAddress = String.format("http://%s:%s/%s", serverAddress, port, apiUrl);
int timeoutInMs = timeout;
HttpClientBuilder builder = HttpClientBuilder.create();
RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(timeoutInMs).build();
builder.setDefaultRequestConfig(requestConfig);
client = builder.build();
}
public ActivemqBrokerInfo(String serverAddress) {
this(serverAddress, DEFAULT_PORT, DEFAULT_API_URL, DEFAULT_TIMEOUT);
}
#Override
public boolean testConnection() {
HttpGet theHttpGet = new HttpGet(serverAddress);
theHttpGet.addHeader("test-header-name", "test-header-value");
HttpResponse httpResponse = null;
try{
httpResponse = client.execute(theHttpGet);// Code is crashing on this line
} catch (IOException ex){
LOGGER.error("Broker down: ", ex);
}
return httpResponse != null;
}
When ActiveMQ runs is normally starts an embedded web server. This web server is used to host the web admin console as well as the Jolokia endpoint which acts as an HTTP facade in front of the broker's MBeans. In other words, any client can send HTTP requests to specially formed URLs on the broker to get results from the underlying management beans. This is exactly what your bit of code appears to be doing. It appears to be sending an HTTP request to the Jolokia endpoint (i.e. api/jolokia) in order to determine if the broker is alive or not.
Based on the information provided it is impossible to determine why testConnection() is not returning successfully since you've included no information about the configuration or state of the broker.
I recommend you add additional logging to see what may be happening and also catch Exception rather than just IOException.
I have an MVC website that makes an API call using a HttpClient. I configured the timeout to be 2 minutes and the API call generally returns within 90 seconds. I can see from application logging that the API completes its execution. If I also call the API method using a tool like Postman or SOAPUI I get a JSON response after 90 seconds.
I cannot get my MVC application to return the response and instead after 2 minutes it will throw an OperationCanceledException which I catch.
public async Task<CommonApiResult> PostAsync<T>(string url, TimeSpan timeout, T payload)
{
using (var client = new HttpClient())
{
client.Timeout = timeout;
var response = await client.PostAsJsonAsync(url, payload);
if (response.IsSuccessStatusCode)
{
var stringContent = await response.Content.ReadAsString(url).ConfigureAwait(false);
return new CommonApiResult<T>
{
StatusCode = response.StatusCode,
Data = stringContent.Deserialize<T>(url)
};
}
}
return null;
}
I never get to the code if (response.IsSuccessStatusCode).
Is there something obvious I am doing wrong? I have looked at a lot of questions and they all seem to be incorrectly using. Result and not using await like me.
just in case someone visit this link in the future.
if it is called via like button click event please check does the button in form tag and has type submit. if yes remove it from the tag
So, my problem is that I'm trying to stream audio on Xamarin.iOS with the AudioFileStream and OutputAudioQueue classes. I have added handlers for the PacketDecoded and PropertyFound events, but they are not being triggered. What's wrong? My code is below...
class AudioStreamer : IAudioStreamer // this is my dependency service interface
{
bool outputStarted;
AudioFileStream afs;
OutputAudioQueue oaq;
public void StartStreaming(string url)
{
afs = new AudioFileStream(AudioFileType.MP3);
// event handlers, these are never triggered
afs.PacketDecoded += OnPacketDecoded;
afs.PropertyFound += OnPropertyFound;
GetAudio(url);
}
void GetAudio(string url)
{
// HTTP
NSUrlSession session = NSUrlSession.FromConfiguration(
NSUrlSessionConfiguration.DefaultSessionConfiguration,
new SessionDelegate(afs),
NSOperationQueue.MainQueue);
var dataTask = session.CreateDataTask(new NSUrl(url));
dataTask.Resume();
}
// event handler - never executed
void OnPropertyFound(object sender, PropertyFoundEventArgs e)
{
if(e.Property == AudioFileStreamProperty.ReadyToProducePackets)
{
oaq = new OutputAudioQueue(afs.StreamBasicDescription);
oaq.BufferCompleted += OnBufferCompleted;
}
}
// another event handler never executed
void OnPacketDecoded(object sender, PacketReceivedEventArgs e)
{
IntPtr outBuffer;
oaq.AllocateBuffer(e.Bytes, out outBuffer);
AudioQueue.FillAudioData(outBuffer, 0, e.InputData, 0, e.Bytes);
oaq.EnqueueBuffer(outBuffer, e.Bytes, e.PacketDescriptions);
// start playing if not already
if(!outputStarted)
{
var status = oaq.Start();
if (status != AudioQueueStatus.Ok)
System.Diagnostics.Debug.WriteLine("Could not start audio queue");
outputStarted = true;
}
}
void OnBufferCompleted(object sender, BufferCompletedEventArgs e)
{
oaq.FreeBuffer(e.IntPtrBuffer);
}
}
// instantiated in GetAudio()
class SessionDelegate : NSUrlSessionDataDelegate
{
readonly AudioFileStream afs;
public SessionDelegate(AudioFileStream afs)
{
this.afs = afs;
}
// this is, too, never executed
public override void DidReceiveData(NSUrlSession session, NSUrlSessionDataTask dataTask, NSData data)
{
afs.ParseBytes((int)data.Length, data.Bytes, false);
}
}
Btw, I mostly copied the code from this screencast.
Your code looks correct...
1) I see the "http://" comment in your source. Are your streaming from a http:// source without adding an ATS exception? If so, the dataTask.Resume(); is a "silent code failure" and thus none of your delegates/events will be called, BUT iOS will log it.
Check output log for something like:
StreamingAudio[13602:591658] App Transport Security has blocked a cleartext HTTP (http://) resource load since it is insecure. Temporary exceptions can be configured via your app's Info.plist file.
Use https:// sources, but if you have to turn ATS off completely for testing:
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key><true/>
</dict>
Note: Search iOS ATS for more info or details in order to only allow non-secure http:// from the server(s) that you would be streaming from, but the iOS' future is https:// only so start preparing... ;-)
2) If the URL source provided to NSUrlSession produces a Connection refused, that is a silent failure, AND there is no logging about it and of course none of your delegates/events are called... Pre-test that your streaming web services are active, try using curl, wget, etc.. to test the stream...
3) I see your streaming MP3s, so you should be OK, but remember there are hundreds of variations to the format and iOS does not stream/play them all so try a different mp3 encoded via a different toolset just to double-check...