Integration test with testcontainers (Redis) Failing to run - docker

I'm building integration test infrastructure for my micro-service (Java Spring).
Having an issue with testcontainers, as I try to create base class for tests that will start redis on container and will function as redis for the tested service.
the abstract test looks like this:
#SpringBootTest(classes = Application.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#ActiveProfiles(Constants.SPRING_PROFILE_DEVELOPMENT)
public class AbstractRedisContainerTest {
#Rule
public GenericContainer redis = new GenericContainer("redis:3.0.6")
.withExposedPorts(6379);
And my log show the follow fail msg:
2020-03-04 12:28:55.545 ERROR [myService,,,] 27709 --- [main] o.t.d.DockerClientProviderStrategy:
Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
2020-03-04 12:28:55.546 ERROR [myService,,,] 27709 --- [main] o.t.d.DockerClientProviderStrategy:
EnvironmentAndSystemPropertyClientProviderStrategy:
failed with exception InvalidConfigurationException (ping failed)
2020-03-04 12:28:55.546 ERROR [myService,,,] 27709 --- [main] o.t.d.DockerClientProviderStrategy:EnvironmentAndSystemPropertyClientProviderStrategy:
failed with exception InvalidConfigurationException (ping failed)
2020-03-04 12:28:55.546 ERROR [myService,,,] 27709 --- [main] o.t.d.DockerClientProviderStrategy:UnixSocketClientProviderStrategy:
failed with exception InvalidConfigurationException (ping failed). Root cause LastErrorException ([13])
2020-03-04 12:28:55.546 ERROR [myService,,,] 27709 --- [main] o.t.d.DockerClientProviderStrategy: ProxiedUnixSocketClientProviderStrategy:
failed with exception InvalidConfigurationException (ping failed). Root cause TimeoutException (null)
2020-03-04 12:28:55.546 ERROR [myService,,,] 27709 --- [main] o.t.d.DockerClientProviderStrategy: As no valid configuration was found, execution cannot continue
org.testcontainers.containers.ContainerLaunchException: Container startup failed
where this is the error for the failing line in the code:
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image:
RemoteDockerImage(imageNameFuture=java.util.concurrent.CompletableFuture#3186b07d[Completed normally],
imagePullPolicy=DefaultPullPolicy(), dockerClient=LazyDockerClient.INSTANCE)
Any idea how to configure my env or what missing here?
important to state that I have succeed to locally run docker images (the one stated here as well) and I have functioning docker environment from cli. I run this test from idea.
not sure if that related, but I run it as user (not as root).

Related

How to fix springboot error Failed to retrieve RMIServer stub: javax.naming.CommunicationException error during JRMP connection establishment

I am having issue with springboot maven project and cassandra in docker.
I want to execute mvn verify and in pre-integration-test to start cassandra docker and api server, execute test and then in post-integration-test to stop server but I am facing error Could not contact Spring Boot application: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment;
I have created github repo so you can easily reproduce error.
Steps to reproduce:
git clone https://github.com/gandra/docker-cassandra-with-initial-seed.git
cd docker-cassandra-with-initial-seed
mvn verify
this will produce error like this:
[INFO]
[INFO] --- spring-boot-maven-plugin:2.2.2.RELEASE:start (api-server-start) # docker-cassandra-with-initial-seed ---
[INFO] Attaching agents: []
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.501 s
[INFO] Finished at: 2021-01-21T18:39:05+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.2.2.RELEASE:start (api-server-start) on project docker-cassandra-with-initial-seed: Could not contact Spring Boot application: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
[ERROR] java.io.EOFException]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Any idea how to fix.
p.s.
I am working on mac and this solution would work on mac and linux as well.
Code https://github.com/gandra/docker-cassandra-with-initial-seed is fixed and now working.
Basically 2 thing are changed:
Removed port mapping 9001 from docker-cassandra.sh.
Added sleep 20 seconds in cassandra_start function in docker-cassandra.sh in order to give time cassandra to properly boot up. Not nice but at least working.
Now followiung code should work with 1 integration test executing successfully:
git clone https://github.com/gandra/docker-cassandra-with-initial-seed.git
cd docker-cassandra-with-initial-seed
mvn verify
If you are using Windows, there was an issue solved in 2020 that is a little bit tricky to find (here the complete thread on Github if you want to read the full story).
Just to recap, the solution is to fix the jmxPort. For example:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<jmxPort>9011</jmxPort>
</configuration>
[...]
The problem here is that he tries to use a port that is already in use.
I had a similar problem with spring boot (not with cassandra).
Two ways i see to solving this:
First way:
Find the program that is using that port (in my case it was "Intel(R) Graphics Command Center" on port 9001). (you can use netstat /a /b in command prompt) or use tcpview (external free program)
disable the program
Profit
Second way
configuring the jmx port in your pom on the spring-boot-maven-plugin
(Answer from sixro)
BUT, i don't like this way because you basically force the others in your project to use your workaround, while the issue is on your machien.

Issue with Docker container deployment via elastic beanstalk : basex 9.3.1

I have this Dockerfile :
FROM basex/basexhttp:9.3.1
I deploy it on Elasticbeanstalk but i get the following error :
[main] INFO org.eclipse.jetty.util.log - Logging initialized #549ms to
org.eclipse.jetty.util.log.Slf4jLog [main] INFO
org.eclipse.jetty.server.Server - jetty-9.4.24.v20191120; built:
2019-11-20T21:37:49.771Z; git:
363d5f2df3a8a28de40604320230664b9c793c16; jvm 1.8.0_212-b04 [main]
WARN org.eclipse.jetty.webapp.WebAppContext - Failed startup of
context
o.e.j.w.WebAppContext#31a5c39e{/,null,UNAVAILABLE}{/srv/basex/webapp}
java.lang.IllegalStateException: Parent for temp dir not configured
correctly: writeable=false
at org.eclipse.jetty.webapp.WebInfConfiguration.makeTempDirectory(WebInfConfiguration.java:499)
at org.eclipse.jetty.webapp.WebInfConfiguration.resolveTempDirectory(WebInfConfiguration.java:468)
at org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure(WebInfConfiguration.java:138)
at org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:488)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:523)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:407)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:100)
at org.eclipse.jetty.server.Server.doStart(Server.java:371)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.basex.BaseXHTTP.(BaseXHTTP.java:129)
at org.basex.BaseXHTTP.main(BaseXHTTP.java:53) [main] INFO org.eclipse.jetty.server.AbstractConnector - Started
ServerConnector#48eff760{HTTP/1.1,[http/1.1]}{0.0.0.0:8984} [main]
INFO org.eclipse.jetty.server.Server - Started #936ms
java.lang.NullPointerException
and then container stops.
Now i tried to use the same image and run it via the root user and it starts and works perfectly fine. Following is the command :
docker run -d b58e4a50371d
Any suggestions/help appreciated.

Elasticbeanstalk Deployment Fails

I am trying to deploy a docker based environment that is using 1-1 application load balancer in AWS elasticbeanstalk. I am getting the below error message. It seemed to deploy very well before but for unknown reasons the new deploys fail.
2019-01-31 18:52:23 INFO Environment update is starting.
2019-01-31 18:53:03 INFO Deploying new version to instance(s).
2019-01-31 18:53:12 INFO Successfully pulled node:10.15.0
2019-01-31 18:57:34 INFO Successfully built aws_beanstalk/staging-app
2019-01-31 18:57:42 ERROR Failed to start nginx, abort deployment
2019-01-31 18:57:44 ERROR [Instance: i-0d4b3811324b90c27] Command failed on instance. Return code: 1 Output: nginx: [emerg] no host in upstream ":8080" in /etc/nginx/conf.d/elasticbeanstalk-nginx-docker-upstream.conf:2
nginx: configuration file /etc/nginx/nginx.conf test failed
Failed to start nginx, abort deployment.
Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2019-01-31 18:57:45 INFO Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2019-01-31 18:57:45 ERROR Unsuccessful command execution on instance id(s) 'i-0d4b3811324b90c27'. Aborting the operation.
2019-01-31 18:57:45 ERROR Failed to deploy application.
This issue was caused of an Uncaught error in my Node js application. Issue is not with Beanstalk.

Weblogic not coming up inside the Dockder container

Weblogic is not coming up . It is giving following stack trace . Can any one help in solving that ?
<Jun 20, 2018 1:04:27,029 PM UTC> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: A MultiException has 4 exceptions. They are:
1. java.lang.ExceptionInInitializerError
2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.rjvm.RJVMService
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.protocol.ProtocolRegistrationService errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.protocol.ProtocolRegistrationService
A MultiException has 4 exceptions. They are:
1. java.lang.ExceptionInInitializerError
2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.rjvm.RJVMService
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.protocol.ProtocolRegistrationService errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.protocol.ProtocolRegistrationService
at org.jvnet.hk2.internal.Collector.throwIfErrors(Collector.java:89)
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:250)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:358)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:487)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:305)
Truncated. see log file for complete stacktrace
Caused By: java.lang.ExceptionInInitializerError
at weblogic.utils.net.AddressUtils.getIPForLocalHost(AddressUtils.java:163)
at weblogic.rjvm.JVMID.setLocalID(JVMID.java:278)
at weblogic.rjvm.RJVMService.setJVMID(RJVMService.java:72)
at weblogic.rjvm.RJVMService.start(RJVMService.java:54)
at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:76)
Truncated. see log file for complete stacktrace
Caused By: java.lang.NullPointerException
at weblogic.utils.net.AddressUtils$AddressMaker.getAllAddresses(AddressUtils.java:62)
at weblogic.utils.net.AddressUtils$AddressMaker.<clinit>(AddressUtils.java:45)
at weblogic.utils.net.AddressUtils.getIPForLocalHost(AddressUtils.java:163)
at weblogic.rjvm.JVMID.setLocalID(JVMID.java:278)
at weblogic.rjvm.RJVMService.setJVMID(RJVMService.java:72)
Truncated. see log file for complete stacktrace
>
The WebLogic Server encountered a critical failure
Reason: Assertion violated
Stopping Derby server...
Derby server stopped.
Actually there was an interface resolution problem inside the docker container which was causing this
Make sure following points for resolution :
1) Cat /etc/hosts should have entry corresponding to localhost
2) docker0 interface should be in up state

Visual Studio Test Run: Run Coded UI automated tests from test plans in the Test hub

Build Definition
I am using the same machine as Build and Release configuration. I have created build successfully. Using build I am able to execute Coded UI scripts in Visual Studio Test Task in build and they are working fine My configuration is for build definition is mentioned below
Release Destination
after successful build definition and execution of test scripts, My next plan is to Run Automated Tests from Test plans in the Test Hub. I have associated my test scripts with Test cases also. Please have a look at the image of my release definition where I have selected Test Run using the Test run
Notification I receive after failed execution of automated test from test plan in test hub is
Deployment of release Release-11 Rejected in Deploy Test Scripts.
Log
2018-02-21T14:24:20.8978238Z AgentName: EVSRV017-DEVSRV017-4
2018-02-21T14:24:20.8978238Z AgentId: 29
2018-02-21T14:24:20.9038250Z ServiceUrl: https://mytfsserver/tfs/DefaultCollection/
2018-02-21T14:24:20.9038250Z TestPlatformVersion:
2018-02-21T14:24:20.9038250Z EnvironmentUri: dta://env/Calculator/_apis/release/16/20/1
2018-02-21T14:24:20.9038250Z QueryForTaskIntervalInMilliseconds: 3000
2018-02-21T14:24:20.9038250Z MaxQueryForTaskIntervalInMilliseconds: 10000
2018-02-21T14:24:20.9048252Z QueueNotFoundDelayTimeInMilliseconds: 3000
2018-02-21T14:24:20.9058254Z MaxQueueNotFoundDelayTimeInMilliseconds: 50000
2018-02-21T14:24:20.9058254Z RetryCountWhileConnectingToTfs: 3
2018-02-21T14:24:20.9058254Z ===========================================
2018-02-21T14:24:21.3909224Z Initializing the Test Execution Engine
Warning
2018-02-21T14:25:02.1240674Z ##[warning]Failure attempting to call the restapis. Exception: System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
2018-02-21T14:25:02.1240674Z at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
2018-02-21T14:25:02.1240674Z at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2018-02-21T14:25:02.1240674Z --- End of inner exception stack trace ---
ERROR:
2018-02-22T10:10:42.0007605Z ##[error]System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
I added debug variable and it seems that when creating a setting file, it is something like system.io exception.
Enabled DEBUG LOG
2018-02-22T20:17:53.8151287Z Initializing the Test Execution Engine
2018-02-22T20:17:53.8161287Z ##[debug]Creating test settings. test settings name : 44de4d5b-f134-4ba2-b0de-ebd8d30b4d22
2018-02-22T20:18:35.3911287Z ##[warning]Failure attempting to call the restapis. Exception: System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
2018-02-22T20:18:35.3931287Z ##[debug]Processed: ##vso[task.logissue type=warning;]Failure attempting to call the restapis. Exception: System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
TFS Agent Log
[INFO VstsAgentWebProxy] No proxy setting found.
[INFO ConfigurationStore] IsServiceConfigured: False
[INFO ConfigurationManager] Is service configured: False
Worker Log
[2018-02-23 19:32:17Z INFO VstsAgentWebProxy] No proxy setting found.
[2018-02-23 19:32:52Z INFO JobServerQueue] Try to append 1 batches web console lines, success rate: 1/1.
[2018-02-23 19:32:52Z INFO JobServerQueue] Try to append 1 batches web console lines, success rate: 1/1.
[2018-02-23 19:32:53Z INFO JobServerQueue] Try to append 1 batches web console lines, success rate: 1/1.
[2018-02-23 19:33:34Z INFO JobServerQueue] Catch exception during update timeline records, try to update these timeline records next time.
[2018-02-23 19:33:34Z INFO ProcessInvoker] Finished process with exit code 0, and elapsed time 00:00:49.0055812.
[2018-02-23 19:33:34Z INFO StepsRunner] Step result: Failed
[2018-02-23 19:33:34Z INFO StepsRunner] Update job result with current step result 'Failed'.
[2018-02-23 19:33:34Z INFO StepsRunner] Current state: job state = 'Failed'
[2018-02-23 19:33:34Z INFO JobRunner] Job result after all job steps finish: Failed
[2018-02-23 19:33:34Z INFO JobRunner] Run all post-job steps.
[2018-02-23 19:33:34Z INFO JobRunner] Job result after all post-job steps finish: Failed
[2018-02-23 19:33:34Z INFO JobRunner] Completing the job execution context.
[2018-02-23 19:33:34Z INFO JobServerQueue] Try to append 2 batches web console lines, success rate: 2/2.
[2018-02-23 19:33:34Z INFO JobRunner] Shutting down the job server queue.
[2018-02-23 19:33:34Z ERR JobServerQueue] Microsoft.VisualStudio.Services.Common.VssServiceException: String or binary data would be truncated.
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponse(HttpResponseMessage response)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__45`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__27`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__26`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Agent.JobServerQueue.<ProcessTimelinesUpdateQueueAsync>d__32.MoveNext()
[2018-02-23 19:33:34Z INFO JobServerQueue] Fire signal to shutdown all queues.
[2018-02-23 19:33:35Z INFO JobServerQueue] All queue process task stopped.
[2018-02-23 19:33:35Z INFO JobServerQueue] Try to append 1 batches web console lines, success rate: 1/1.
[2018-02-23 19:33:35Z INFO JobServerQueue] Web console line queue drained.
[2018-02-23 19:33:35Z INFO JobServerQueue] Try to upload 2 log files or attachments, success rate: 2/2.
[2018-02-23 19:33:35Z INFO JobServerQueue] File upload queue drained.
[2018-02-23 19:33:35Z INFO JobServerQueue] Timeline update queue drained.
[2018-02-23 19:33:35Z INFO JobServerQueue] All queue process tasks have been stopped, and all queues are drained.
[2018-02-23 19:33:35Z INFO JobRunner] Raising job completed event.
[2018-02-23 19:33:35Z INFO Worker] Job completed.
I will be thankful to you if anyone can identify where I am missing something or what I need to fix this issue so that I can execute automated tests from test plans in the test hub.
Regards
Based on the error message "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host." and your clarification. It should be related to the known issue on win server 2008 R2. Please refer to below article for details:
Team Foundation Server: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
However the bug has been fixed by the Windows team and they have
released a QFE for it. You can find the QFE here. You will
need to install it on all of your ATs.
So, just try to install the hotfix and restart the computer after you apply this hotfix, then try it again.
You can also try to use the initial workarounds that list in the blog:
Open the IIS Manager
In the Connections pane, make sure the name of your AT is selected.
In the middle pane (titled “ Home”), make sure you are
in the “Features View” (bottom) and scroll down to the Management
section.
Double-click the “Configuration Editor” icon.
The middle pane should now have the title “Configuration Editor”.
In the Section pull down near the top, expand the
system.applicationHost and select “webLimits”.
You should now see a bunch of property value pairs, one of which is
named “minBytesPerSecond”. Its value is most like 240. You will
want to lower this value for the workaround.
Besides, another possibility is that it's caused by the Proxy server, just try to bypass the proxy server, then check it again.

Resources