I have a simple question regarding the architecture of my Amazon Simple Workflow / AWS Flow for Ruby app. For background, I have a simple workflow with one activity running in an AWS Flow for Ruby layer on Opsworks. I have a separate REST API running in a Rails App Server layer on Opsworks that I would like to kick off the workflow.
The code in the REST API that kicks off the workflow:
1: domain = AWS::SimpleWorkflow.new.domains['my_domain']
2: workflow_client = AWS::Flow::workflow_client(domain.client, domain) {{from_class: MyWorkflowClass}}
3: workflow_client.start_execution(input_1: #input1, input_2: #input2)
My assumption is that my workflow and REST API code bases could be separate and that the only common component would be the aws-flow Ruby gem and require 'aws/decider'. However, I'm finding that my REST API also needs to have require 'PATH_TO_MY_WORKFLOW_CLASS'. When I remove that line of code from the code file in my REST API that kicks off the workflow, I get the following error:
undefined method `_options' for nil:NilClass; ["/Users/MyName/.rvm/gems/ruby-2.0.0-p247/gems/aws-flow-2.2.1/lib/aws/decider/utilities.rb:183:in `interpret_block_for_options'", "/Users/MyName/.rvm/gems/ruby-2.0.0-p247/gems/aws-flow-2.2.1/lib/aws/decider/implementation.rb:73:in `workflow_client'"
(error at line 2 above)
Am I mistaken? Do I really need to require MyWorkflowClass in my workflow starter app (i.e. my REST API) or am I doing something wrong? I've scoured the documentation and could not find a clear answer to this. All the samples that I can find do indeed have the workflow class included in the workflow starter code, but I'm not sure if it's because they are bundled as a simple sample or if it's because it's the way it's supposed to be. The reason why I am not taking the samples at face value is because requiring the workflow class in the workflow starter code does not make any sense to me. It binds the two apps way too tightly for my taste.
I posted an issue on the aws-flow-ruby sdk and got the answer from an Amazon Engineer. In short, you can use the :from_class option or the :prefix_name and :execution_method options together.
There are two ways of starting the workflow in code
1) Using the aws sdk directly.
In this case, your code doesn't need to know anything about the workflow class. You just need the domain, workflow type (name and version) and the workflow id.
It will look something like -
require 'aws-sdk-v1'
swf = AWS::SimpleWorkflow.new.client
swf.start_workflow_execution(
domain: "HelloWorld",
workflow_type: {
name: "HelloWorldWorkflow",
version: "1.0"
},
workflow_id: "foo",
input: ....,
....other options (optional)...
)
As you can see above, this doesn't require the workflow class at all.
2) Using the aws-flow gem (which is what you are doing above).
There are two ways of using the workflow client provided by the aws-flow gem to start an execution. You can either use the client as a generic client and not tie it to any workflow class or you can use the :from_class option to fetch options from a particular workflow class. To use the from_class option, you need to have the class in the ObjectSpace (hence you need to require the workflow file).
With from_class -
require 'aws/decider'
domain = AWS::SimpleWorkflow.new.domains['my_domain']
workflow_client = AWS::Flow::workflow_client(domain.client, domain) {{from_class: "MyWorkflowClass"}}
workflow_client.start_execution(input_1: #input1, input_2: #input2)
Without from_class -
require 'aws/decider'
domain = AWS::SimpleWorkflow.new.domains['my_domain']
workflow_client = AWS::Flow::workflow_client(domain.client, domain) {{
prefix_name: "YourClassName",
execution_method: "workflow_method_name",
version: "1.0",
...other options...
}}
workflow_client.start_execution(input_1: #input1, input_2: #input2)
The recommended way to start a workflow execution is to use aws-flow WorkflowClient instead of using the SDK directly.
Additional notes with respect to the input accepted by a workflow:
The SDK and the console will only take strings as input. This can be a free form string but if your workflow is written using ruby flow, this string should be a serialized form of your input so that the WorkflowWorker can deserialize the input when it picks up the task and convert it into ruby objects (in this case a hash).
When you use the ruby flow WorkflowClient, the client will automatically serialize your input hash (or any other input) into a string before sending it to SWF. aws-flow by default uses a YAML based data converter to do this (It can be overridden).
If you just want to see what your input hash will look like as a string, you can do the following -
AWS::Flow::FlowConstants.default_data_converter.dump(input_hash)
You can then use this serialized input to start a workflow using the SDK or the console.
Related
I have developed a rest-api client (in java) customised to the needs of my product. I wanted to generate tests using my rest api client using swagger-codegen modules based on yaml-file.
I have already extended DefaultCodegenConfig & even tried implementing the CodegenConfig interface to build my custom jar. I have customized the api.mustache and api_test.mustache files and passing them in the constructor and processOpts() method of my CustomCodeGen that extends DefaultCodegenConfig.
However, I want to use the custom/new mustache template variables that I have added in my customised api.mustache.
For e.g. if refer to standard api.mustache, the template variables it typically uses are
- {{classname}}
- {{#operation}}
- {{#contents}}
- {{#parameters}}
etc.
Now, I want to introduce a new template variable, let's say {{custom_param}}. Now I am not clear how do I integrate this new template variable with the implementation.
Looks like from this Mustache-Template-Variables published here, swagger-codegen does not allow adding new template-variables and perhaps we are restricted to only the variables mentioned on this page.
So, is there some way to make the new template variables work ?
Some time ago I added the uniqueItems parameter for bean validation as it was not getting processed by the engine even though it was a part of the implemented JSR.
So I believe codebase needs to be updated to use your own variable which is only possible if you fork the code.
In case it helps, these two were the PRs:
For query parameters: https://github.com/swagger-api/swagger-codegen/pull/10154.
For body parameters: https://github.com/swagger-api/swagger-codegen/pull/10490.
How do you automatically parse ansible warnings and errors in your jenkins pipeline jobs?
I greatly enjoy the power of leveraging in ansible in jenkins when it works. Upon a failure, the hunt to locate the actual error can be challenging.
I use WarningsNG which supports custom parsers (and allows their programmatic generation)
Do you know of any plugins or addons that already transform these logs into the kind charts similar to WarningsNG?
I figured I'd ask as I go off into deep regex land and make my own.
One good way to achieve this seems to be the following:
select an existing structured output ansible callback plugin (json, junit and yaml are all viable) . I selected junit as I can play with the format to get a really nice view into the playbook with errors reported in a very obvious way.
fork that GPL file (yes, so be careful with that license) to augment with the following:
store output as file
implement the missing callback methods (the three mentioned above do not implement the v2...item callbacks.
forward events to the default or debug callback to ensure operators see something when they execute the plan
add a secrets cleaner - if you use jenkins credentials-binding-plugin it will hide secrets from the console, it will not not hide secrets within stored files. You'll need to handle that in your playbook or via some groovy code (if groovy, try{...} finally { clean } seems a good pattern)
Snippet - forewarding to default callback
from ansible.plugins.callback.default import CallbackModule as CallbackModule_default
...
class CallbackModule(CallbackBase):
CALLBACK_VERSION = 2.0
CALLBACK_TYPE = 'stdout'
CALLBACK_NAME = 'json'
def __init__(self, display=None):
super(CallbackModule, self).__init__(display)
self.default_callback = CallbackModule_default()
...
def v2_on_file_diff(self, result):
self.default_callback.v2_on_file_diff(result)
... do whatever you'd want to ensure the content appears in the json file
I have access to the Code base (checked out to my local machine from SVN). It is written using Java and Groovy using Grails framework (MVC architecture). I am a tester and as part of automating my tests, I want to write code that will make calls to the controllers and in return i can check the result in terms of looking at response or entries in data base. I basically want to skip the UI part.
How can I start? I probably cannot write my code inside the dev project (i am not allowed to i suppose). Do i need to create a separate framework for it? Or can I take all the jar files and include then in a project and write code on top of it?
The answer in this post is actually what I am looking for but for a Java application. Is there any API i can use?
Please let me know if you need additional information.
If the application does not provide Json, XML or similar APIs, you can use a test library like HtmlUnit within jUnit test methods.
A example from "Getting Started" section:
#Test
public void homePage_Firefox() throws Exception {
final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);
final HtmlPage page = webClient.getPage("http://htmlunit.sourceforge.net");
Assert.assertEquals("HtmlUnit - Welcome to HtmlUnit", page.getTitleText());
webClient.closeAllWindows();
}
Note that HtmlUnit tries to work like a virtual browser (written 100% in Java), but it is a bit limited in executing Javascript, for example.
Then, use another library like jsoup or Jericho HTML Parese to inspect the code and get the values you want to check in the database.
In the other hand, if the application does provide methods to obtain the data, you can use Jersey Client API to make REST requests and get the values. It is very simple. Look at this example:
Client client = ClientBuilder.newClient();
WebTarget target = client.target("http://localhost:9998").path("resource");
Form form = new Form();
form.param("x", "foo");
form.param("y", "bar");
MyJAXBBean bean =
target.request(MediaType.APPLICATION_JSON_TYPE)
.post(Entity.entity(form,MediaType.APPLICATION_FORM_URLENCODED_TYPE),
MyJAXBBean.class);
I have a question about ruby and wsdl soap.
I couldn't find a way to get each method's params and their type.
For example, if I found out that a soap has a methods called "get_user_information" (using wsdlDriver) is there a way to know if this method requires some params and what type of params does it require (int, string, complex type, ecc..)?
I'd like to be able to build html forms from a remote wsdl for each method...
Sorry for my horrible English :D
Are you using soapr4?
Soap4r comes with a command line client to build proxies for accessing web services via SOAP. This is preferable to using the wsdlDriver which has to build the proxy dynamically every time it runs.
To build a "permanent" proxy then you need to run the following command
wsdl2ruby.rb --type client --wsdl http://some/path/to/the/wsdl
When this command runs then you should end up with a bunch of ruby files one of which (probably default.rb) will call each method in turn and document the necessary inputs and outputs.
Alternatively you may find the Wsdl Analyser useful. This will allow you to enter the URL for a WSDL which it will then analyse and list all of the operations and (sometimes) the paramaters required
Thank you for the very quick response!
I'll try to explain myself a little better :D
I've tried soap4r, and I'm able to get soap's methods with something like this:
require "soap/wsdlDriver"
client = SOAP::WSDLDriverFactory.new(my-wsdl-url).create_rpc_driver
puts client.singleton_methods
What I'd like to know is:
If, for example, my soap has a method called "get_some_params_and_sum_them", is there a way to know how many params it takes and which type they should be?
Something like
puts client.method("get_some_params_and_sum_them").params
Wsdl Analyser does it, and I'd like to know if this is possible also in a ruby script without tons of code lines :D
I need to implement a web app, but instead of using relational database I need to use different SOAP Web Services as a back-end. An important part of application only calls web services and displays the result. Since Web Services are clearly defined in form of Operation: In parameters and Return Type it seems to me that basic GUI could be easily constructed just like in the case of scaffolding based on Domain Entities.
For example in case of SearchProducts web service operation I need to enter search parameters as input, so the search page can be constructed. Operation will return a list of products, so I need a page that will display this list in some kind of table.
Is there already some library in grails that let you achieve this. If not, how would you go about creating one?
Probably the easiest approach is to use wsimport on the WSDL files to generate the client-side stubs. Then you can call methods in the stubs from Groovy just as you would have called them from Java.
For example, consider the WSDL file for Microsoft's TerraServer, located at http://terraservice.net/TerraService.asmx?wsdl . Then you run something like
wsimport -d src -keep http://terraservice.net/TerraService.asmx?WSDL
which puts all the compiled stubs in the src directory. Then you can write Groovy code like
import com.terraserver_usa.terraserver.*;
TerraServiceSoap sei = new TerraService().getTerraServiceSoap()
Place home = new Place(city:'Boston',state:'MA',country:'US')
def pt = sei.convertPlaceToLonLatPt(home)
println "$pt.lat, $pt.lon"
assert Math.abs(pt.lat - 42.360000) < 0.001
assert Math.abs(pt.lon - -71.05000) < 0.001
If you want to access a lot of web services, generate the stubs for all of them. Or you can use dynamic proxies instead.
The bottom line, though, is to let Java do what it already does well, and use Groovy where it makes your life easier.
You should be able to use XFire or CXF Plugins. For automatic scaffolding, modify your Controller.groovy template in scaffolding templates so it auto-generates methods you need.