How to configure testcontainers to run docker with gradle? - docker

For my project I need to have a task that starts a test container, but when I am trying to start my task I have the following error
Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration
I think it's a configuration issue.
I am using kotlin gradle, my build file looks like this :
import org.testcontainers.containers.PostgreSQLContainer
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
testImplementation("org.testcontainers:postgresql")
}
buildscript {
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
"classpath"("org.testcontainers:postgresql:1.17.5")
}
}
tasks.register("postgresContainer") {
val containerInstance = PostgreSQLContainer<Nothing>("postgres:12.8")
.apply {
withDatabaseName("test")
}
containerInstance.start()
extra["containerUrl"] = containerInstance.jdbcUrl
extra["containerUsername"] = containerInstance.username
extra["containerPassword"] = containerInstance.password
extra["containerDatabaseName"] = containerInstance.databaseName
extra["containerInstance"] = containerInstance
}
I have omited most of it but cannot find what I am missing
I tried to refer to the documentation of test containers but cannot find my specific case, docker seems to be correctly configured, I think the error comes from my build.kts

I would suggest the following.
Add this to your build.gradle.kts file to add support for test containers (this specific example is for PostgreSQL but the idea is the same...):
dependencies {
testImplementation("org.testcontainers:junit-jupiter")
testImplementation("org.testcontainers:postgresql:$testContainerVersion")
}
dependencyManagement {
imports {
mavenBom("org.testcontainers:testcontainers-bom:$testContainerVersion")
}
}
Add the following to your tests class as a global variable:
val postgreSQLContainer = PostgreSQLContainer<Nothing>("postgres:14")
.apply {
withReuse(true)
start()
}
Personally I've added that into an AbstractIntegrationTest class and also configured the container there as follow:
abstract class AbstractIntegrationTest {
companion object {
#JvmStatic
#DynamicPropertySource
fun properties(registry: DynamicPropertyRegistry) {
registry.add("spring.datasource.url", postgreSQLContainer::getJdbcUrl)
registry.add("spring.datasource.username", postgreSQLContainer::getUsername)
registry.add("spring.datasource.password", postgreSQLContainer::getPassword)
}
}
}
Note that the withReuse(true) is just an optimization that ensures that the DB is not re-created for each test as this is a heavy operation, and therefore you need to programmatically truncate your tables between tests...
For instance:
#BeforeEach
fun setUp() {
// Test util class that truncate all tables
repoUtils.cleanTables()
}

Related

How to get #Testcontainers(disabledWithoutDocker = true) behaviour in junit4/testcontainers

My codebase is ancient and is locked into JUnit4. I would like to integrate testcontainers with the project to incorporate docker containers in the automated tests.
My dev box (which I control) runs docker, however, my CI system (which I do not control) does not.
If I could use JUnit5, I would just add the #Testcontainers(disabledWithoutDocker = true) annotation, and the docker based test would run happily on my dev box, while being disabled on the CI machine.
What is the JUnit4 equivalent to #Testcontainers(disabledWithoutDocker = true)?
I'm not sure if there is something out-of-the-box for this in Testcontainers for JUnit 4. You can mirror the JUnit 5 feature with some custom code.
First, you need a way to conditionally execute tests. There is already a good answer for this available. Basically you use JUnit 4 Assumptions for this:
#Before
public void beforeMethod() {
org.junit.Assume.assumeTrue(someCondition());
// rest of setup.
}
You will need this for all your Docker related tests.
Next, someCondition() should evaluate whether Docker is available or not. The current Testcontainers (1.14.3) release uses the following code part for #Testcontainers(disabledWithoutDocker = true):
private ConditionEvaluationResult evaluate(Testcontainers testcontainers) {
if (testcontainers.disabledWithoutDocker()) {
if (isDockerAvailable()) {
return ConditionEvaluationResult.enabled("Docker is available");
}
return ConditionEvaluationResult.disabled("disabledWithoutDocker is true and Docker is not available");
}
return ConditionEvaluationResult.enabled("disabledWithoutDocker is false");
}
boolean isDockerAvailable() {
try {
DockerClientFactory.instance().client();
return true;
} catch (Throwable ex) {
return false;
}
}
So you could extract isDockerAvailable() to e.g. an abstract class that also includes the #Before and handle this for yourself:
public abstract class DockerizedTest {
#Before
public void shouldRunTest() {
org.junit.Assume.assumeTrue(isDockerAvailable());
}
boolean isDockerAvailable() {
try {
DockerClientFactory.instance().client();
return true;
} catch (Throwable ex) {
return false;
}
}
}
and now all your Docker related tests can extend DockerizedTest. Whenever the assumption evaluates to false, the test will be ignored.
If #Before is too late, you can use the same approach with #BeforeClass.

How to read a workspace file from within Jenkins Shared Library?

I have a Jenkins pipeline, it imports a shared library which in turn "grabs" a dependency. I want to call a method on the class which takes a File parameter.
But the file doesn't exist when I try to access it in the shared library.
The Pipeline
#Library('my shared library')
pipeline {
// ...
stages {
stage {
steps {
script {
f = new File("foo.txt")
libFn(myFile: f)
}
}
}
}
}
The Jenkins Shared Library
// ./vars/libFn.groovy
#Grab("3rd party java library")
import LibraryClass
def call(Map params) {
LibraryClass libraryClass = new LibraryClass();
libraryClass.processFile(params.myFile);
}
The 3rd Party Java Library
public class LibraryClass {
public void processFile(File file) {
// do something with a file
}
}

how to run a complete Jenkins scripted pipeline stage from shared libraries src file

I recently started with Jenkins shared libraries in Jenkins pipeline.
I created a "func.groov" class and located it under "src/org/prj/func.groovy" :
package org.prj
import jenkins.model.
class func implements Serializable {
def steps
func(steps) {
this.steps = steps
}
def sh(args) {
steps.sh "echo ${args}"
}
def setLBL(CurrentNodeName,newLabelName){
jenkins.model.Jenkins.instance.slaves.each{ slave ->
if (slave.getNodeName() == CurrentNodeName){
slave.setLabelString(newLabelName)
}
}
}
Jenkinsfile (scripted) looks like:
#Library('prj') import org.prj.func
def utils = new func(steps)
node(lbl)
{
stage("A"){
Build_node_lbl = env.NODE_NAME+System.currentTimeMillis()
utils.setLBL(env.NODE_NAME,Build_node_lbl)
}
}
so currently it works. my question is how to create a full stage (like "A") as a function in func.groovy shared lib which will include, for example:
GIT checkout step
sh compilation step
Artifactory deploy step
Im actually looking to create a "building blocks" (a "Build" in my example) with Jenkins pipeline and shard libraries.
1. With Class Instantiation
You can create a class like you would do in Java. Then in your Jenkinsfile you instantiate the class and call its function.
src/org/prj/MyPipeline.groovy:
package org.prj
class MyPipeline {
def steps
MyPipeline(steps) {this.steps = steps}
public def build() {
steps.node('lbl') {
steps.stage('A') {
// Do build stuff
// steps.sh(..)
}
}
}
}
Jenkinsfile:
import org.prj.MyPipeline
def pipeline = new MyPipeline(this)
pipeline.build()
2. With Static Functions
You may also work with static contexts, without instantiation. However, this would require to hand over the caller context to the pipeline:
src/org/prj/MyPipeline.groovy:
package org.prj
class MyPipeline {
public static def build(caller) {
caller.node('lbl') {
caller.stage('A') {
// Do build stuff
caller.sh(..)
}
}
}
}
Jenkinsfile:
import org.prj.MyPipeline
MyPipeline.build(this)

grails jms start listening after apllication is completly launched

I'm using grails jms-1.3plugin and I have the problem, that my jms listener Service starts consuming messages from activeMQ before the application is fully up and running. This results in an error when I try to write some messages to the DB.
So my question is, how can I manage to start consuming from a queue manually. So that I can set autoStartup to false.
here is my example grails code:
ConsumerService.groovy
package jmsstartstop
import grails.plugin.jms.Queue
class ConsumerService {
static exposes = ["jms"]
#Queue(name="liesMich")
def receiveMessage(String msg) {
log.info("Received Message:" + msg)
}
}
resources.groovy
import org.apache.activemq.ActiveMQConnectionFactory
import org.springframework.jms.connection.SingleConnectionFactory
beans = {
jmsConnectionFactory(SingleConnectionFactory) {
targetConnectionFactory = { ActiveMQConnectionFactory cf ->
brokerURL = grailsApplication.config.jms.brokerURL
}
}
}
Config.groovy
jms{
brokerURL='tcp://localhost:61616'
containers {
standard {
autoStartup = false
}
}
}
What I'm looking for is something like jmsConnectionFactory.getTargetConnectionFactory().start() that can be called in Bootstrap.groovy or maybe in a controller manually. But unfortunately this start method does not exist in the TargetConnectionFactory.
Is there a way to do it, or any other suggestions?
Bootstrap.groovy (which is not working)
class BootStrap {
def jmsConnectionFactory
def init = { servletContext ->
jmsConnectionFactory.??WHATEVER??.start()
}
def destroy = {
}
}
The issue is that the plugin starts processing messages before the Datasource plugin (part of Grails) has finished it's own startup.
The good news is that this appears to be fixed in the latest SNAPSHOT version of the plugin.
To use the SNAPSHOT change your plugin as such: :jms:1.3-SNAPSHOT in your BuildConfig.groovy
What worked for me is to MANUALLY start the JMSListener services on Bootstrap file:
e.g.
In listener.groovy:
class ClientListenerService {
boolean transactional = true
static exposes = ["jms"]
static destination = "com.moviesxd.api.domain.Client_QUEUE"
static isTopic = false
static container = "manualStart"
In bootstrap.groovy:
def clientRequestListenerJmsListenerContainer
...
clientRequestListenerJmsListenerContainer.start()
This solves the problem.

Grails scheduling a task without plugins

For one of our applications we have a different Tasks that we would like to happen on a scheduled basis. However we don't want to bother with quartz for several different reasons.
In grails, how do we go about scheduling a task that can run on a regular basis?
After researching for quite some time we came to this conclusion:
Within the Groovy Source Packages we created an interface
interface Task{
void executeTask()
}
Next we created our Task:
class SayHelloTask implements Task{
void executeTask(){
println "Hello"
}
}
Within the resources.groovy file we added the following:
import package.SayHelloTask
beans = {
sayHelloTask(SayHelloTask){
}
xmlns task: "http://www.springframework.org/schema/task"
task.'scheduled-tasks'{
task.scheduled(ref:'retryEmailTask', method: 'executeTask', cron: '0-59 * * * * *')
}
}
We went with this solution because it cut the overhead of Quartz. It matches how we do things in our Java projects.
I prefer using the annotations on my services when dealing with Spring based scheduled tasks.
grails-app/conf/spring/resrouces.groovy
beans {
xmlns task: "http://www.springframework.org/schema/task"
task.'annotation-driven'('proxy-target-class': true)
}
Then on my service:
class MyService {
#Scheduled(cron="*/5 * * * * MON-FRI")
void doSomething() {
...
}
}
Regardless of how you do this, be cautious about your Hibernate session scope. Good luck!
For the records, as of Grails 3.2.10 this can be achieved neatly by using annotations the following way.
Create an ordinary Grails service:
class ScheduledService {
boolean lazyInit = false // <--- this is important
#Scheduled(fixedRate = 20000L)
def myBusinessMethodForTheJob() {
log.info 'Executing scheduled job...'
}
}
Enable scheduling in the application:
#EnableScheduling
class Application extends GrailsAutoConfiguration {
static void main(String[] args) {
GrailsApp.run(Application, args)
}
}
Done.
Another option is the Timer and TimerTask classes provided by the JDK. You can run this example in the Groovy console to see it in action
def task = new TimerTask() {
void run() {
println "task running at ${new Date()}"
}
}
def firstExecutionDelay = 1000
def delayBetweenExecutions = 2000
new Timer(true).schedule(task, firstExecutionDelay, delayBetweenExecutions)

Resources