I am using a class that has below method, how do I get and parse or save the Steam from this?
void SomeMethod(Action<Stream> stream)
Related
How to access the elements of a side input if I have my class extend DoFn?
For example:
Say I have a ParDo transform like:
PCollection<String> data = myData.apply("Get data",
ParDo.of(new MyClass()).withSideInputs(myDataView));
And I have a class:-
static class MyClass extends DoFn<String,String>
{
//How to access side input here
}
c.sideInput() isn't working in this case.
Thanks.
In this case, the problem is that the processElement method in your DoFn does not have access to the PCollectionView instance in your main method.
You can pass the PCollectionView to the DoFn in the constructor:
class MyClass extends DoFn<String,String>
{
private final PCollectionView<..> mySideInput;
public MyClass(PCollectionView<..> mySideInput) {
// List, or Map or anything:
this.mySideInput = mySideInput;
}
#ProcessElement
public void processElement(ProcessContext c) throws IOException
{
// List or Map or any type you need:
List<..> sideInputList = c.sideInput(mySideInput);
}
}
You would then pass the side input to the class when you instantiate it, and indicate it as a side input like so:
p.apply(ParDo.of(new MyClass(mySideInput)).withSideInputs(mySideInput));
The explanation for this is that when you use an anonymous DoFn, the process method has a closure with access to all the objects within the scope that encloses the DoFn (among them is the PCollectionView). When you're not using an anonymous DoFn, there is no closure, and you need another way of passing the PCollectionView.
So although the answer above is correct, it is still a little incomplete.
So once you finish implementing the above answer, you need to execute your pipeline like this:
p.apply(ParDo.of(new MyClass(mySideInput)).withSideInputs(mySideInput));
I'm trying to write a library that will make it easer for dartisans to use the SoundCloud JavaScript SDK (http://developers.soundcloud.com/docs/api/sdks#javascript).
I'm using the 'dart:js' library, and
I'm only using one class to handle the proxy.
class SCproxy {
JsObject proxy = context['SC'];
String client_id;
SCproxy(this.client_id) {}
initialize() {
proxy.callMethod('initialize', [client_id]);
}
stream(String track_id){
var track = new JsObject(proxy.callMethod('stream',[track_id]));
print(track); // track should be the soundmanager2 object that we can call '.play' on.
}
The repo I'm hosting this from is (https://github.com/darkkiero/scproxy)
My problem occurs when I try to run my 'stream' method.
main() {
SCproxy SC = new SCproxy('Your SoundCloud API client_ID');
SC.initialize();
SC.stream('/tracks/111477464');
}
When I try to grab and use the soundmanager2 object returned by the javascript 'SC.stream' method, the dart editor gives me this exception :
Breaking on exception: type 'ScriptElement' is not a subtype of type 'JsFunction' of 'constructor'.
I am under the impression that I should be able to get the dart JsObject for the soundmanager2 object by collecting the callback of the 'SC.stream', But I'm not sure how.However I could be completely misusing 'dart:js' and that would be helpful information as well.
You don't seem to follow the SoundCloud JavaScript SDK documentation. Particularly for the stream method that takes a callback as parameter and doesn't return.
The following Dart code :
context['SC'].callMethod('stream', ['/tracks/293', (sound) {
sound.callMethod('play');
}]);
will do the same as this JS code :
SC.stream("/tracks/293", function(sound){
sound.play();
});
You can have a look at Using JavaScript from Dart for more explanations.
I've made a SAX parser for parsing XML files with a number of different tags. For performance reasons, I chose SAX over DOM. And I'm glad I did because it works fast and good. The only issue I currently have is that the main class (which extends DefaultHandler) is a bit large and not very easy on the eyes. It contains a huge if/elseif block where I check the tag name, with some nested if's for reading specific attributes. This block is located in the StartElement method.
Is there any nice clean way to split this up? I would like to have a main class which reads the files, and then a Handler for every tag. In this Tag Handler, I'd like to read the attributes for that tag, do something with them, and then go back to the main handler to read the next tag which again gets redirected to the appropriate handler.
My main handler also has a few global Collection variables, which gather information regarding all the documents I parse with it. Ideally, I would be able to add something to those collections from the Tag Handlers.
A code example would be very helpful, if possible. I read something on this site about a Handler Stack, but without code example I was not able to reproduce it.
Thanks in advance :)
I suggest setting up a chain of SAX filters. A SAX filter is just like any other SAX Handler, except that it has another SAX handler to pass events into when it's done. They're frequently used to perform a sequence of transformations to an XML stream, but they can also be used to factor things the way you want.
You don't mention the language you're using, but you mention DefaultHandler so I'll assume Java. The first thing to do is to code up your filters. In Java, you do this by implementing XMLFilter (or, more simply, by subclassing XMLFilterImpl)
import java.util.Collection;
import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.XMLFilterImpl;
public class TagOneFilter extends XMLFilterImpl {
private Collection<Object> collectionOfStuff;
public TagOneFilter(Collection<Object> collectionOfStuff) {
this.collectionOfStuff = collectionOfStuff;
}
#Override
public void startElement(String uri, String localName, String qName,
Attributes atts) throws SAXException {
if ("tagOne".equals(qName)) {
// Interrogate the parameters and update collectionOfStuff
}
// Pass the event to downstream filters.
if (getContentHandler() != null)
getContentHandler().startElement(uri, localName, qName, atts);
}
}
Next, your main class, which instantiates all of the filters and chains them together.
import java.util.ArrayList;
import java.util.Collection;
import org.xml.sax.XMLReader;
import org.xml.sax.helpers.XMLReaderFactory;
public class Driver {
public static void main(String[] args) throws Exception {
Collection<Object> collectionOfStuff = new ArrayList<Object>();
XMLReader parser = XMLReaderFactory.createXMLReader();
TagOneFilter tagOneFilter = new TagOneFilter(collectionOfStuff);
tagOneFilter.setParent(parser);
TagTwoFilter tagTwoFilter = new TagTwoFilter(collectionOfStuff);
tagTwoFilter.setParent(tagOneFilter);
// Call parse() on the tail of the filter chain. This will finish
// tying the filters together before executing the parse at the
// XMLReader at the beginning.
tagTwoFilter.parse(args[0]);
// Now do something interesting with your collectionOfStuff.
}
}
With the release of the new dart:io libraries, in order to read the data from an HttpRequest we now need to:
Listen to the body to handle data and notified of it's completion.
In order to accomplish this, I've been using something similar to the following to get all of the data from the request (taken from the dart:io tests):
List<int> body = new List<int>();
request.listen(body.addAll, onDone: () {
var str = new String.fromCharCodes(body);
// Now do something with the string of data
});
Is there some way of using transform and/or StringDecoders to already provide a constructed result. The only way I can think of, I would still need to create a StringBuffer and use writeAll to ensure all data is passed in the event it doesn't all arrive at once, and still call the onDone before using the string in the buffer.
So I guess ultimately the question is: is there some way I can use a HttpRequest (or in all actuality any Stream) without needing to buffer/build the results and can just pass a callback or handler which receives the entire contents?
I haven't tried this, but I think you should be able to use StringDecoder, toList() and join() to get the whole body:
request.transform(new StringDecoder()).toList().then((data) {
var body = data.join('');
print(body);
}
I was wondering if I can capture the result of an action after the result returns and the JSP is rendered. I want to be able to take the entire result (generated HTML) and push it into memcached so I can bring it via Nginx with-out hitting the application server. Any ideas?
PS: I know I can run the interceptor after the action executes but before the result returns and the JSP is rendered, but not after the JSP is rendered.
I haven't found a way to do this inside of struts2, your best bet it to create a servlet Filter and have it modify the OutputStream.
http://onjava.com/pub/a/onjava/2003/11/19/filters.html
Hey I know its quite late now to answer you might have already found out the answer, however for others to benefit I am posting the answer.
One thing that is very similar to what you are doing is done by sitemesh filter.
Yes, filter comes before and after the Struts2 filter itself, so you can mess with the inputs and outputs easily.
But struts does evaluate JSP/freemarker/velocity and generate the final html which is passed to the user. JSP is a bit trickey because internally a servlet is called but check out org.apache.struts2.views.freemarker.FreemarkerResult class, you can see the actual html getting generated in template.process(model, writer);. This writer is actually ServletActionContext.getResponse().getWriter();
Now to get the html all you need to do is
ServletActionContext.getResponse().getWriter().toString() //This does not work out of box. To get the toString() to work you need to use a ResponseWrapper - which is the same method to get result html in Filters. See- Programming Customized Requests and Responses.
Listing to modify resulting html in struts 2. This is not tested, but it is extracted from my code I have written earlier for custom template engine. I will probably post full description in Custom template engine for struts2
public class DecoratorInterceptor implements Interceptor {
public String intercept(ActionInvocation invocation) throws Exception {
final ActionContext context = invocation.getInvocationContext ();
HttpServletResponse responseParent = (HttpServletResponse)
context.get(ServletActionContext.HTTP_RESPONSE);
CharResponseWrapper wrapper = new CharResponseWrapper(responseParent);
ServletActionContext.setResponse(wrapper);
//Actual Action called
String result = invocation.invoke();
String htmlReturned = wrapper.toString();
//play with htmlReturned ...
String modifiedhtml = pushintoMemCache(htmlReturned );
CharArrayWriter car = new CharArrayWriter();
car.write(modifiedhtml );
PrintWriter out = responseParent.getWriter();
out.write(car.toString());
out.flush();
}
#Override
public void destroy() {
// TODO Auto-generated method stub
}
#Override
public void init() {
// TODO Auto-generated method stub
}
}
Read this article - http://struts.apache.org/2.0.6/docs/interceptors.html
SUMMARY:When you request a resource
that maps to an "action", the
framework invokes the Action object.
But, before the Action is executed,
the invocation can be intercepted by
another object. After the Action
executes, the invocation could be
intercepted again. Unsurprisingly, we
call these objects "Interceptors."
Question: How do you determine if the view has been generated? Do you set a request header or an some sort of a flag to determine if the view has been generated?
You could try throwing a MemCachedException to indicate that it is time to load into a mem cache. Your interceptor code could read
try {
return invocation.invoke();
} catch (MemCachedException mce) {
// Your code to upload to MemCache.
} finally {
// blah blah clean up.
}
Within your interceptor's intercept() method, the ActionInvocation parameter has a getResult() method which returns null before Action execution (i. e. before you call invocation.invoke() in your intercept() method) and contains an implementation of Result afterwards. That object should give you some way to access the data you need, but how this is done probably depends on the class that is actually used.
See also my somewhat related question and the answer I posted after figuring it out.