How to get #Testcontainers(disabledWithoutDocker = true) behaviour in junit4/testcontainers - docker

My codebase is ancient and is locked into JUnit4. I would like to integrate testcontainers with the project to incorporate docker containers in the automated tests.
My dev box (which I control) runs docker, however, my CI system (which I do not control) does not.
If I could use JUnit5, I would just add the #Testcontainers(disabledWithoutDocker = true) annotation, and the docker based test would run happily on my dev box, while being disabled on the CI machine.
What is the JUnit4 equivalent to #Testcontainers(disabledWithoutDocker = true)?

I'm not sure if there is something out-of-the-box for this in Testcontainers for JUnit 4. You can mirror the JUnit 5 feature with some custom code.
First, you need a way to conditionally execute tests. There is already a good answer for this available. Basically you use JUnit 4 Assumptions for this:
#Before
public void beforeMethod() {
org.junit.Assume.assumeTrue(someCondition());
// rest of setup.
}
You will need this for all your Docker related tests.
Next, someCondition() should evaluate whether Docker is available or not. The current Testcontainers (1.14.3) release uses the following code part for #Testcontainers(disabledWithoutDocker = true):
private ConditionEvaluationResult evaluate(Testcontainers testcontainers) {
if (testcontainers.disabledWithoutDocker()) {
if (isDockerAvailable()) {
return ConditionEvaluationResult.enabled("Docker is available");
}
return ConditionEvaluationResult.disabled("disabledWithoutDocker is true and Docker is not available");
}
return ConditionEvaluationResult.enabled("disabledWithoutDocker is false");
}
boolean isDockerAvailable() {
try {
DockerClientFactory.instance().client();
return true;
} catch (Throwable ex) {
return false;
}
}
So you could extract isDockerAvailable() to e.g. an abstract class that also includes the #Before and handle this for yourself:
public abstract class DockerizedTest {
#Before
public void shouldRunTest() {
org.junit.Assume.assumeTrue(isDockerAvailable());
}
boolean isDockerAvailable() {
try {
DockerClientFactory.instance().client();
return true;
} catch (Throwable ex) {
return false;
}
}
}
and now all your Docker related tests can extend DockerizedTest. Whenever the assumption evaluates to false, the test will be ignored.
If #Before is too late, you can use the same approach with #BeforeClass.

Related

How to configure testcontainers to run docker with gradle?

For my project I need to have a task that starts a test container, but when I am trying to start my task I have the following error
Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration
I think it's a configuration issue.
I am using kotlin gradle, my build file looks like this :
import org.testcontainers.containers.PostgreSQLContainer
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
testImplementation("org.testcontainers:postgresql")
}
buildscript {
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
"classpath"("org.testcontainers:postgresql:1.17.5")
}
}
tasks.register("postgresContainer") {
val containerInstance = PostgreSQLContainer<Nothing>("postgres:12.8")
.apply {
withDatabaseName("test")
}
containerInstance.start()
extra["containerUrl"] = containerInstance.jdbcUrl
extra["containerUsername"] = containerInstance.username
extra["containerPassword"] = containerInstance.password
extra["containerDatabaseName"] = containerInstance.databaseName
extra["containerInstance"] = containerInstance
}
I have omited most of it but cannot find what I am missing
I tried to refer to the documentation of test containers but cannot find my specific case, docker seems to be correctly configured, I think the error comes from my build.kts
I would suggest the following.
Add this to your build.gradle.kts file to add support for test containers (this specific example is for PostgreSQL but the idea is the same...):
dependencies {
testImplementation("org.testcontainers:junit-jupiter")
testImplementation("org.testcontainers:postgresql:$testContainerVersion")
}
dependencyManagement {
imports {
mavenBom("org.testcontainers:testcontainers-bom:$testContainerVersion")
}
}
Add the following to your tests class as a global variable:
val postgreSQLContainer = PostgreSQLContainer<Nothing>("postgres:14")
.apply {
withReuse(true)
start()
}
Personally I've added that into an AbstractIntegrationTest class and also configured the container there as follow:
abstract class AbstractIntegrationTest {
companion object {
#JvmStatic
#DynamicPropertySource
fun properties(registry: DynamicPropertyRegistry) {
registry.add("spring.datasource.url", postgreSQLContainer::getJdbcUrl)
registry.add("spring.datasource.username", postgreSQLContainer::getUsername)
registry.add("spring.datasource.password", postgreSQLContainer::getPassword)
}
}
}
Note that the withReuse(true) is just an optimization that ensures that the DB is not re-created for each test as this is a heavy operation, and therefore you need to programmatically truncate your tables between tests...
For instance:
#BeforeEach
fun setUp() {
// Test util class that truncate all tables
repoUtils.cleanTables()
}

How can I inject with Guice my api into dataflow jobs without needed to be serializable?

This question is a follow on after such a great answer Is there a way to upload jars for a dataflow job so we don't have to serialize everything?
This made me realize 'ok, what I want is injection with no serialization so that I can mock and test'.
Our current method requires our apis/mocks to be serialiable BUT THEN, I have to put static fields in the mock because it gets serialized and deserialized creating a new instance that dataflow uses.
My colleague pointed out that perhaps this needs to be a sink and that is treated differently? <- We may try that later and update but we are not sure right now.
My desire is from the top to replace the apis with mocks during testing. Does someone have an example for this?
Here is our bootstrap code that does not know if it is in production or inside a feature test. We test end to end results with no apache beam imports in our tests meaning we swap to any tech if we want to pivot and keep all our tests. Not only that, we catch way more integration bugs and can refactor without rewriting tests since the contracts we test are customer ones we can't easily change.
public class App {
private Pipeline pipeline;
private RosterFileTransform transform;
#Inject
public App(Pipeline pipeline, RosterFileTransform transform) {
this.pipeline = pipeline;
this.transform = transform;
}
public void start() {
pipeline.apply(transform);
pipeline.run();
}
}
Notice that everything we do is Guice Injection based so the Pipeline may be direct runner or not. I may need to modify this class to pass things through :( but anything that works for now would be great.
The function I am trying to get our api(and mock and impl to) with no serialization is thus
private class ValidRecordPublisher extends DoFn<Validated<PractitionerDataRecord>, String> {
#ProcessElement
public void processElement(#Element Validated<PractitionerDataRecord>element) {
microServiceApi.writeRecord(element.getValue);
}
}
I am not sure how to pass in microServiceApi in a way that avoid serialization. I would be ok with delayed creation as well after deserialization using guice Provider provider; with provider.get() if there is a solution there too.
Solved in such a way that mocks no longer need static or serialization anymore by one since glass bridging the world of dataflow(in prod and in test) like so
NOTE: There is additional magic-ness we have in our company that passes through headers from service to service and through dataflow and that is some of it in there which you can ignore(ie. the RouterRequest request = Current.request();). so for anyone else, they will have to pass in projectId into getInstance each time.
public abstract class DataflowClientFactory implements Serializable {
private static final Logger log = LoggerFactory.getLogger(DataflowClientFactory.class);
public static final String PROJECT_KEY = "projectKey";
private transient static Injector injector;
private transient static Module overrides;
private static int counter = 0;
public DataflowClientFactory() {
counter++;
log.info("creating again(usually due to deserialization). counter="+counter);
}
public static void injectOverrides(Module dfOverrides) {
overrides = dfOverrides;
}
private synchronized void initialize(String project) {
if(injector != null)
return;
/********************************************
* The hardest part is this piece since this is specific to each Dataflow
* so each project subclasses DataflowClientFactory
* This solution is the best ONLY in the fact of time crunch and it works
* decently for end to end testing without developers needing fancy
* wrappers around mocks anymore.
***/
Module module = loadProjectModule();
Module modules = Modules.combine(module, new OrderlyDataflowModule(project));
if(overrides != null) {
modules = Modules.override(modules).with(overrides);
}
injector = Guice.createInjector(modules);
}
protected abstract Module loadProjectModule();
public <T> T getInstance(Class<T> clazz) {
if(!Current.isContextSet()) {
throw new IllegalStateException("Someone on the stack is extending DoFn instead of OrderlyDoFn so you need to fix that first");
}
RouterRequest request = Current.request();
String project = (String)request.requestState.get(PROJECT_KEY);
initialize(project);
return injector.getInstance(clazz);
}
}
I suppose this may not be what you're looking for, but your use case makes me think of using factory objects. They may depend on the pipeline options that you pass (i.e. your PipelineOptions object), or on some other configuration object.
Perhaps something like this:
class MicroserviceApiClientFactory implements Serializable {
MicroserviceApiClientFactory(PipelineOptions options) {
this.options = options;
}
public static MicroserviceApiClient getClient() {
MyPipelineOptions specialOpts = options.as(MySpecialOptions.class);
if (specialOpts.getMockMicroserviceApi()) {
return new MockedMicroserviceApiClient(...); // Or whatever
} else {
return new MicroserviceApiClient(specialOpts.getMicroserviceEndpoint()); // Or whatever parameters it needs
}
}
}
And for your DoFns and any other execution-time objects that need it, you would pass the factory:
private class ValidRecordPublisher extends DoFn<Validated<PractitionerDataRecord>, String> {
ValidRecordPublisher(MicroserviceApiClientFactory msFactory) {
this.msFactory = msFactory;
}
#ProcessElement
public void processElement(#Element Validated<PractitionerDataRecord>element) {
if (microServiceapi == null) microServiceApi = msFactory.getClient();
microServiceApi.writeRecord(element.getValue);
}
}
This should allow you to encapsulate the mocking functionality into a single class that lazily creates your mock or your client at pipeline execution time.
Let me know if this matches what you want somewhat, or if we should try to iterate further.
I have no experience with Guice, so I don't know if Guice configurations can easily pass the boundary between pipeline construction and pipeline execution (serialization / submittin JARs / etc).
Should this be a sink? Maybe, if you have an external service, and you're writing to it, you can write a PTransform that takes care of it - but the question of how you inject various dependencies will remain.

Is this loginRequired(f)() the way to handle login required functions in dart?

I am new to Dart programming. I am trying to figure out what is the proper way (what everyone will do) to handle/guard those functions which are login required. The following is my first trial:
$ vim login_sample.dart:
var isLoggedIn;
class LoginRequiredException implements Exception {
String cause;
LoginRequiredException(this.cause);
}
Function loginRequired(Function f) {
if (!isLoggedIn) {
throw new LoginRequiredException("Login is reuiqred.");
}
return f;
}
void secretPrint() {
print("This is a secret");
}
void main(List<String> args) {
if (args.length != 1) return null;
isLoggedIn = (args[0] == '1') ? true : false;
try {
loginRequired(secretPrint)();
} on LoginRequiredException {
print("Login is required!");
}
}
then, run it with $ dart login_sample.dart 1 and $ dart login_sample.dart 2.
I am wondering if this is the recommended way to guard login required functions or not.
Thank you very much for your help.
Edited:
My question is more about general programming skills in Dart than how to use a plugin. In python, I just need to add #login_required decorator in the front of the function to protect it. I am wondering if this decorator function way is recommended in dart or not.
PS: All firebase/google/twitter/facebook etc... are blocked in my country.
I like the functional approach. I'd only avoid using globals, you can wrap it in a Context so you can mock then for tests and use Futures as Monads: https://dartpad.dartlang.org/ac24a5659b893e8614f3c29a8006a6cc
Passing the function is not buying much value. In a typical larger Dart project using a framework there will be some way to guard at a higher level than a function - such as an entire page or component/widget.
If you do want to guard at a per-function level you first need to decide with it should be the function or the call site that decides what needs to be guarded. In your example it is the call site making the decision. After that decision you can implement a throwIfNotAuthenticated and add a call at either the definition or call site.
void throwIfNotAuthenticated() {
if (!userIsAuthenticated) {
throw new LoginRequiredException();
}
}
// Function decides authentication is required:
void secretPrint() {
throwIfNotAuthenticated();
print('This is a secret');
}
// Call site decides authentication is required:
void main() {
// do stuff...
throwIfNotAuthenticated();
anotherSecreteMethod();
}

Navigate to Specflow step from within Step Definitions

So I have a few tests where i've reused steps from within steps.
But i'm now having a nightmare on the maintenance front, in that I cant easily navigate between the steps.
Here's an example:
[Given(#"I have an order")]
public void GivenIHaveAnOrder()
{
Given("an open store");
Given("I am an existing customer");
Given("I am on homepage");
When("I search for a component");
When("I add the component to my basket");
}
How do I navigate to one of those internal steps?
If I wanted to navigate to the "When("I search for a component");" step I cant.
If I were on the feature file I could simply right click the step and "go to definition" but I cant do that here. Does anyone have a solution?
I assume that you are calling the steps with the Given/When- functions, because they are in a different binding class. Am I right?
There is a better way of doing it, than using this functions.
Did you had a look at the driver concept and context injection?
Have a look here: http://www.specflow.org/documentation/Context-Injection/
Simply extract your logic of your steps to a driver class and get an instance from it in the different step classes:
class Driver
{
public void AnOpenStore()
{
...
}
}
[Binding]
public class StepClass1
{
private Driver _driver;
public StepClass1(Driver driver)
{
_driver = driver;
}
[Given(#"I have an order")]
public void IHaveAnOrder()
{
_driver.AnOpenStore();
}
}
[Binding]
public class StepClass2
{
private Driver _driver;
public StepClass2(Driver driver)
{
_driver = driver;
}
[Given(#"an open store")]
public void AnOpenStore()
{
_driver.AnOpenStore();
}
}
When you arrange your step implementations like that, the reusing of other steps is much more easier.

How to debug static code block in GEB Page model

I am trying out GEB and wanted to debug the static code block in the examples. I have tried to set breakpoints but i seem unable to inspect the data that is used in the static content block.
class GoogleResultsPage extends Page {
static at = { results }
static content = {
results(wait: true) { $("li.g") }
result { i -> results[i] }
resultLink { i -> result(i).find("a.l")[0] }
firstResultLink { resultLink(0) }
}
}
Any clue on how this normally can be debugged using for example IntelliJ?
Since the content block is using a DSL and undergoes a transformation when compiled I'm thinking it wouldn't be possible to debug without special support from the IDE, however I hope someone can prove me wrong.
The approach I have been using is to define methods for anything beyond the core content. This provides a few benefits, including debugging support, IDE autocompletion when writing tests, and good refactoring support. The drawback of course is slightly more verbose code, although the tradeoff has been worth it for my purposes.
Here's how I might do the GoogleResultsPage:
class GoogleResultsPage extends Page {
static at = { results }
static content = {
results(wait: true) { $("li.g") }
}
Navigator result(int i) { results[i] }
Navigator resultLink(int i) { result(i).find("a.l")[0] }
Navigator firstResultLink { resultLink(0) }
}
Then when writing the test I use a slightly more typed approach:
class MySpec extends GebReportingSpec {
def "google search with keyword should have a first result"() {
given:
GoogleHomePage homePage = to(GoogleHomePage)
when:
homePage.search("keyword")
then:
GoogleResultsPage resultsPage = at(GoogleResultsPage)
resultsPage.result(0).displayed
}
}

Resources