I have set
spring.cloud.deployer.local.workingDirectoriesRoot=D:\\Deploy
in application.properties to define log folder.
However, It does not works, the log of task when I launched is written on java.io.tmpdir (C:/..../temp/).
I am using spring-cloud-dataflow-server-core version 2.3.0.RELEASE
Does I need to put these config in dataflow-server.yml ?
How can I configure the log folder of SCDF and their tasks?
We just added some tests for windows and noticed that correct format (assuming task name is timestamp) which actually works on windows when using drive letter is:
deployer.timestamp.local.working-directories-root=file:/C:/tmp
Think is how in a spring world binding works from String to java.nio.file.Path. For those interested to know more, It's spring core's PathEditor which uses Paths.get(URI), thus need for file:.
The property workingDirectoriesRoot is a local deployer property and not an application property. Hence, it needs to be passed when launching the task.
task launch <task-name> --properties "deployer.<task-name>.local.workingDirectoriesRoot=D:\\Deploy"
Related
We are using Spring XD for executing some batch jobs and considering to use Spring Cloud Dataflow. For this I wanted to remote debug a execution of a Task and I was not able to make it working.
I tried to export the following environment variable before the SCDF server is started:
spring.cloud.deployer.local.javaOpts=Xdebug -Xrunjdwp:transport=dt_socket,address=12201,server=y
Also tried to pass as argument in the GUI while invoking the task:
app.<appname>.local.javaOpts=Xdebug -Xrunjdwp:transport=dt_socket,address=12201,server=y
Nothing seems to be working.
I'm able to debug the composed-task-runner launched by SCDF using the listen debugger mode, this will also work for your task as well.
Run Debugger in your IDE in listen mode on port 5006. (this project's classpath should have composed-task-runner sources, put break point some where )
Run SCDF with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 option, attach debugger to the SCDF process in your IDE on port 5005 (attach mode).
Put breakpoint at this line :
String javaOptsString = getValue(deploymentProperties, "javaOpts");
in JavaCommandBuilder class (for spring-cloud-deployer-local v.1.3.0.M2 it's line #83).
Launch your task - debugger stops at breakpoint.
Step Over once in your IDE, the value of javaOptsString is null now. Using IDE, set the value of javaOptsString to
-agentlib:jdwp=transport=dt_socket,server=n,address=localhost:5006,suspend=y
Press Resume in IDE.
Your breakpoint set in #1 should be hit in few seconds.
If you know how to pass javaOpts as deployment properties of your task - you will be able to debug in listen mode without this nightmare ;-). I've not found a way to escape = and , characters in the -agentlib:jdwp=transport=dt_socket,server=n,address=localhost:5006,suspend=y javaOpts deployment property.
We are working on an improved solution for the local-deployer - you can follow spring-cloud/spring-cloud-dataflow#369 for tracking purpose.
There is, however, the following option that exists to aggregate all the application logs into the server console directly, which might be useful while in active development.
stream deploy --name myStream --properties "deployer.*.local.inheritLogging=true"
Finally I was able to remote debug a composed task or regular task. Follow the below steps:
In scdf UI go to tasks and click on the definition section
Click play button (invoke) on the task/composed task that you want to invoke.
On the launch task page define your task arguments
Add the following properties by clicking 'Add Property' button:
- deployer.composed-task-runner.local.debugPort=12103
- deployer.composed-task-runner.local.debugSuspend=y
Now launch the task
You can now see in the log that when the composed task's java process is launched it is called with the debug parameter.
If you want to control the heap memory or any java options you can do by adding the following property:
deployer.composed-task-runner.local.javaOpts=Xmx2048M
Note that 'composed-task-runner' is the name of the App (Not the name of the task).
I have a simple CRUD application with backend code in dropwizard. The entire app just comprises of simple resource classes and crud operations except one case where some business logic is involved.
I am trying to extract this into a service instead of putting it in the resource class itself. But for that my service would need an ondemand jdbi connection to access data and do its thing.
All my connect strings and config values are in YML file. Since this app would be running on different servers with different yml files, I dont want to hardcode the yml file name in order to read it again, to get the connect strings and do it that way.
How do I achieve this?
Can you detect what environment you are on?
If so, can you do something like ${environment}.yml?
There is Configuration project on apache which might help.
Otherwise, is it a case of in dev you want to run
java -jar app.jar server dev.yml
and in prod you want to run java -jar app.jar server prod.yml? I imagine you have separate daemons in each environment. So, those environment's will pick up the right configuration, if you've configured them that way.
Otherwise, if the property names are the same, but their values differ, and you pick up the right yml in the right environment, things should work.
If I haven't addressed your question, can you please elaborate your problem a little more?
I have a Spring Cloud Config app with the Spring Cloud Security dependencies. I'm trying to hit the /encrypt endpoint to encrypt a password.
According to the docs at http://cloud.spring.io/spring-cloud-config/spring-cloud-config.html#_key_management I believe I need to set a symmetric key using "encrypt.key". But I can't figure out where to put this; all combinations I've tried result in {"description":"No key was installed for encryption service","status":"NO_KEY"} when I attempt to POST to /encrypt.
If I POST a key to /key, the /encrypt works perfectly, so I'm pretty sure that everything else is setup correctly. It also works fine using the environment variable ENCRYPT_KEY, or by using a system property encrypt.key. I just can't figure out where to place the encrypt.key within a configuration file. Is there a sample on this?
Thanks #pVilaca, this is indeed what the documentation says, but setting the key in application.properties or application.yml won't work.
Through experimentation I've found that the only place you can set the encrypt.key property is an ENCRYPT_KEY environment variable, a system property, bootstrap.properties, bootstrap.yml, or calling the /key endpoint.
Not sure why application.properties or application.yml don't work in this case. Setting this key must have more of an impact on the startup process than it would appear.
Setting Spring Cloud Version to 'Brixton.SR5' worked for me. For some reason, setting 'encrypt.key' in either application.properties/yml or bootstrap.properties/yml in later Spring Cloud Version does not work anymore.
check properties tag of pom.xml file. If you are using version 2 or 3 of Dalston, change it to:
<spring-cloud.version>Dalston.SR1</spring-cloud.version>
and it should be working fine. In my case i was using version 3 and changing it to version 1 resolved the issue for me.
Reference:
https://github.com/spring-cloud/spring-cloud-config/issues/767
If you're using spring cloud with spring boot, that is the method that is described on the documentation that you mentioned, you've two 'main' properties files.
bootstrap.[properties,yml]
To modify the startup behaviour you can change the location of the config server using bootstrap.properties (like application.properties but for the bootstrap phase of an application context)
application.[properties,yml]
where the "application" is injected as the "spring.config.name" in the SpringApplication (i.e. what is normally "application" in a regular Spring Boot app)
source: Spring Cloud Config Documentation
So, it should be enough to set the encrypt.keyin your application.[properties,yml] file (or the alternative name if specified)
I face the following problem: whenever i try to start Jetty with an application instance i need check if the application is running. I need to do that from an ant target.
I want to have something behaving like the following pseudocode:
**
<target name="target1"
depends="run-jetty-with-application"
description="Target1">
<--when run-jetty-with-application is ok(jetty is up and application is running)-->
<antcall target="target2"/>
<--end when-->
</target>
**
I should also mention that i have no publicly exposed urls that could give me the status of the application.
I'm not after the hacky solution like waitfor.
Thanks in advance!
There are only 2 ways to know if Jetty is started, all webapps are deployed without error, all lifecycle is started successfully, and the server is available to serve content.
Periodically ask for a resource on that web app, confirming that it is up and running. (timeout with failure if resources doesn't make itself available after a period of time)
Hook into the LifeCycle.Listener on the Server component, wait for the .lifeCycleStarted(LifeCycle event) for the server, then walk the entire handler tree, looking for any Handler component that failed to deploy via the DeploymentManager.
As you can imagine, the first technique is the easiest.
This is the <waitfor> facility in Ant, and is what the Jetty project itself uses.
The second approach requires a custom Ant task.
You can either write it yourself, and know that it will do what you want. Or use the jetty-ant tasks with daemon mode to get almost the same thing (the JettyRunTask in daemon mode starts up the server, but it does not wait for start to finish or the webapps to deploy successfully before it returns control to ant).
I have installed the grails web plugin. I can now browse to :
localhost:8080/myappname/console
And I can see the console displayed over there. I have bunch of test-cases written for application. I wish I can test my app from this web console.
Is it possible to do so? I'm very new to grails.
Thanks in advance.
Short answer no. The grails console is meant to write groovy code that interacts with your running application. Your running application does not include your test cases or the grails command line by default.
Long answer sort of. Provided you have all of your projects source code available somewhere in the file system where your application is running, you could call an external process to run test-app and return the result to the user. Here are some docs on running external processes in groovy: http://groovy.codehaus.org/Executing+External+Processes+From+Groovy. I suppose you could also package you application somehow to make this work, but I think doing that would be fairly complex.
I am not sure this is a good use case for the console plugin over all. Hope this helps