I'm trying to write a Grails plugin that records the size of each HTTP response sent by the application it's installed in. The approach I've tried so far is:
(1) Write a servlet filter that wraps the response. The response wrapper decorates the OutputStream with an OutputStream that counts the number of bytes in the response. These classes are shown here.
(2) Register the servlet filter in the plugin descriptor
def doWithWebDescriptor = { webXml ->
def contextParam = webXml.'context-param'
contextParam[contextParam.size() - 1] + {
'filter' {
'filter-name'('countingFilter')
'filter-class'(CountingFilter.name)
}
}
def filter = webXml.'filter'
filter[filter.size() - 1] + {
'filter-mapping' {
'filter-name'('countingFilter')
'url-pattern'('/*')
}
}
}
(3) In a Grails filter, try to retrieve the number of bytes written like so:
afterView = {
// countingFilter registers the response wrapper as an attribute of the request
// named 'counter'
request.getAttribute('counter').byteCount
}
But a value of 0 is always returned. However, I discovered that if I add the code above to both the after and afterView closures of the Grails filter
after = {
// returns 0
request.getAttribute('counter').byteCount
}
afterView = {
// returns the correct response size
request.getAttribute('counter').byteCount
}
It now returns the correct response size, but it seems the content generated by the sitemesh layout has disappeared, leaving only the content in the GSP itself.
So it seems I'm close to a solution. My guess is that I'm flushing buffers at the wrong time, and/or my filter is not running at the correct point in the filter chain.
This is proving to be much more difficult that I expected. Is there a simpler way?
Grails has this feature built in. It's undocumented. Set the system property "GSPResponseWriter.enableContentLength" to true by adding "-DGSPResponseWriter.enableContentLength=true" option to the JVM args. See the source code of GSPResponseWriter for more details.
The reason this isn't enabled by default is that calculating the content length is not simple because some characters are multi-byte in UTF-8 encoding. BoundedCharsAsEncodedBytesCounter is the class doing the calculation.
Related
I'm not a programming savvy person, so please bear with me.
I've read blog entries and docs about command object. I've never used it and was wondering if I should. (I probably should...)
My project requires parsing, sorting, calculating, and saving results into database when users upload files.
So according to one of the blog entries I read and its corresponding github code,
1) SERVICE should receive file uploads, parse uploaded files (mainly docs and pdfs), sort parsed data using RegEx, and calculate data,
2) COMMAND OBJECT should call SERVICE, collect results and send results back to controller, and save results into the database,
3) CONTROLLER should receive request from VIEW, get results from COMMAND OBJECT, and send results back to VIEW.
Did I understand correctly?
Thanks.
I found this to be the best setup. Here is an example that I use on production:
Command Object (to carry data and ensure their validity):
#grails.validation.Validateable
class SearchCommand implements Serializable {
// search query
String s
// page
Integer page
static constraints = {
s nullable: true
page nullable: true
}
}
Controller (directs a request to a Service and then gets a response back from the Service and directs this response to a view):
class SomeController {
//inject service
def someService
def search(SearchCommand cmd) {
def result = someService.search(cmd)
// can access result in .gsp as ${result} or in other forms
render(view: "someView", model: [result: result])
}
}
Service (handles business logic and grabs data from Domain(s)):
class SomeService {
def search(SearchCommand cmd) {
if(cmd.hasErrors()) {
// errors found in cmd.errors
return
}
// do some logic for example calc offset from cmd.page
def result = Stuff.searchAll(cmd.s, offset, max)
return result
}
}
Domain (all database queries are handled here):
class Stuff {
String name
static constraints = {
name nullable: false, blank: false, size: 1..30
}
static searchAll(String searchQuery, int offset, int max) {
return Stuff.executeQuery("select s.name from Stuff s where s.name = :searchQuery ", [searchQuery: searchQuery, offset: offset, max:max])
}
}
Yes, you understood it correctly except the one thing: command object shouldn't save the data to DB - let service to do that. The other advantage of command object is data binding and validation of data from the client. Read more about command objects here grails command object docs
You can also find helpful information regarding your question in this article
grails best practices
I guess not. Its not really related to whether the save is done in a service it should always attempt to carry out complex stuff and specifically db stuff in a service. so that is regardless. I tend to not use command object but have got hooked on helper classes aka beans that sit in src/main/groovy and do all of the validation and formatting. I just did a form and in it has feedback and reason.
Initially I thought I would get away with
def someAction(String feedback, String reason) {
someService.doSomething(feedback,reason)
}
But then I looked closed and my form was firstly a textarea then the selection objects were bytes so above was not working and to simply fix it without adding the complexity to my controller/service I did this:
packe some.package
import grails.validation.Validateable
class SomeBean implements Validateable {
User user
byte reason
String feedback
static constraints = {
user(nullable: true)
reason(nullable:true, inList:UsersRemoved.REASONS)
feedback(nullable:true)
}
void setReason(String t) {
reason=t as byte
}
void setFeedback(String t) {
feedback=t?.trim()
}
}
Now my controller
class SomeController {
def userService
def someService
def doSomething(SomeBean bean){
bean.user = userService.currentUser
if (!bean.validate()) {
flash.message=bean.errors.allErrors.collect{g.message([error : it])}
render view: '/someTemplate', model: [instance: bean,template:'/some/template']
return
}
someService.doSomeThing(bean)
}
}
Now my service
Class SomeService {
def doSomeThing(SomeBean bean) {
if (bean.user=='A') {
.....
}
}
All of that validation would have still had to have been done somewhere, you say no validation but in a good model you should do validation and set things to be stored in proper structures to reduce overloading your db over time. difficult to explain but in short i am talking about your domain class objects and ensuring you are not setting up String something string somethingelse and then not even defining their lenghts etc. be strict and validate
if you have a text area this will be stored in the back end - so you will need to trim it like above - you will need to ensure the input does not exceed the max character of the actual db structure which if not defined will probably be 255
and by doing
static constraints = {
user(nullable: true)
reason(min:1, max:255, nullable:true, inList:UsersRemoved.REASONS)
Has already invalidated it through the bean.validate() in the controller if the user exceeded somehow my front end checks and put in more than 255.
This stuff takes time be patient
Edited to finally add in that example byte - is one to be careful of -
When adding any String or what ever I have started to define the specific like this and in the case of byte if it is a boolean true false - fine if not then define it as a tinyint
static mapping = {
//since there is more than 1 type in this case
reason(sqlType:'tinyint(1)')
feedback(sqlType:'varchar(1000)')
// name(sqlType:'varchar(70)')
}
If you then look at your tables created in the db you should find they have been created as per definition rather than standard 255 varchar which I think is the default for a declared String.
We built an Grails (2.3.7) application where we are overriding the link(Map attr, String encoding = 'UTF-8') in DefaultLinkGenerator class. The reason we are doing this is to have the same URL throughout our application as it was business requirement from the customer. Basically in the overriding link() method, we are modifying the Map with new request parameters.
Now in Production we are seeing the following exception which occurs sporadically and we haven't been able to reproduce it locally.
2014-09-29 01:04:06,257 StackTrace ERROR Full Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: 3
at org.codehaus.groovy.grails.web.mapping.UrlCreatorCache$ReverseMappingKey.<init>(UrlCreatorCache.java:196)
at org.codehaus.groovy.grails.web.mapping.UrlCreatorCache.createKey(UrlCreatorCache.java:62)
at org.codehaus.groovy.grails.web.mapping.DefaultUrlMappingsHolder.getReverseMappingNoDefault(DefaultUrlMappingsHolder.java:265)
at org.codehaus.groovy.grails.web.mapping.DefaultUrlMappingsHolder.getReverseMappingNoDefault(DefaultUrlMappingsHolder.java:257)
at org.codehaus.groovy.grails.web.mapping.DefaultLinkGenerator.link(DefaultLinkGenerator.groovy:213)
at gov.texas.multitenant.core.mapping.MultitenantLinkGenerator.super$2$link(MultitenantLinkGenerator.groovy)
The code in 'UrlCreatorCache$ReverseMappingKey' that throws the above ArrayIndexOutOfException can happen only when Map attribute (params) gets mutated during the loop. The excerpt of that code is below.
paramKeys = new String[params.size()];
paramValues = new String[params.size()];
int i = 0;
for (Map.Entry entry : params.entrySet()) {
**paramKeys[i] = String.valueOf(entry.getKey());** //throws exception here
String value = null;
if (entry.getValue() instanceof CharSequence) {
value = String.valueOf(entry.getValue());
}
...
paramValues[i] = value;
i++;
}
Now my question is, does this Map attribute THREAD SAFE? Can it get mutated between threads since we are modifying it?
Any feedback will be great appreciated. Thanks in advance.
You wouldn't know inside that method - anything implementing the Map interface is allowable. But that's not important - you're causing the problem (by being one of two concurrent editors of an apparently non-threadsafe instance) and can easily fix it.
Instead of modifying the map, create a new one (any Map implementation will do since only your code and the code in that method can access it) and modify that, and pass that in for link generation. Then throw it away and use the real map for further calculations. Something like
def attrs = ... // the 'real' map
def copy = [:] + attrs
copy.foo = 42
copy.bar = 'a very high one indeed'
String link = linkGenerator.link(copy)
//
...
def foo = attrs.foo // not available, set in copy, but no longer using that
...
I messed around with this a bit yesterday and failed miserably. I want to convert:
"/$controller/$action?/$id?"
To
#in psudo
"/$controller/$id?/$action?"
#ideal regex
"\/(\w+)(\/\d+)?(\/\w+)?"
The most obvious way failed "/$controller/$action?/$id?"
I can write the regex's to do it, but I am having trouble finding a way to using true regexs (I found RegexUrlMapping but could not find out how to use it), and also can't find documentation on how to assign a group to a variable.
My question is 2 parts:
How to I define a URL Resource with a true regex.
How to I bind a "group" to a variable. In other words if I define a regex, how do I bind it to a variable like $controller, $id, $action
I would also like to be able to support the .json notation /user/id.json
Other things I have tried, which I thought would work:
"/$controller$id?$action?"{
constraints {
controller(matches:/\w+/)
id(matches:/\/\d+/)
action(matches:/\/\w+/)
}
}
also tried:
"/$controller/$id?/$action?"{
constraints {
controller(matches:/\w+/)
id(matches:/\d+/)
action(matches:/\w+/)
}
}
The grails way to deal with this is to set
grails.mime.file.extensions = true
in Config.groovy. This will cause Grails to strip off the file extension before applying the URL mappings, but make it available for use by withFormat
def someAction() {
withFormat {
json {
render ([message:"hello"] as JSON)
}
xml {
render(contentType:'text/xml') {
//...
}
}
}
For this you'd just need a URL mapping of "$controller/$id?/$action?"
I'm not aware of any way to use regular expressions in the way you want in the URL mappings, but you could get a forward mapping working using the fact that you can specify closures for parameter values that get evaluated at runtime with access to the other params:
"$controller/$a?/$b?" {
action = { params.b ?: params.a }
id = { params.b ? params.a : null }
}
which says "if b is set then use that as the action and a as the id, otherwise use a as the action and set id to null". But this wouldn't give you a nice reverse mapping, i.e. createLink(controller:'foo', action:'bar', id:1) wouldn't generate anything sensible, you'd have to use createLink(controller:'foo', params:[a:1, b:'bar'])
Edit
A third possibility you could try is to combine the
"/$controller/$id/$action"{
constraints {
controller(matches:/\w+/)
id(matches:/\d+/)
action(matches:/\w+/)
}
}
mapping with a complementary
"/$controller/$action?"{
constraints {
controller(matches:/\w+/)
action(matches:/(?!\d+$)\w+/)
}
}
using negative lookahead to ensure the two mappings are disjoint.
I'm trying to generate a sitemap dynamically for a large web site with thousands of pages.
Yes, I have considered generating the sitemap file offline and simply serving it statically, and I might end up doing exactly that. But I think this is a generally useful question:
How can I stream large data from a DB in Wicket?
I followed the instructions at the Wicket SEO page, and was able to get a dynamic sitemap implementation working using a DataProvider. But it doesn't scale- it runs out of memory when it calls my DataProvider's iterator() method with a count arg equal to the total number of objects I'm returning, rather than iterating over them in chunks.
I think the solution lies somewhere with WebResource/ResourceStreamingRequestTarget. But those classes expect an IResourceStream, which ultimately boils down to providing an InputStream implementation, which deals in bytes, rather than DB records. I wouldn't know how to implement the length() method in such a case, as that would require visiting every record ahead of time to compute the overall length.
From the doc of the IResourceStream.length() method:
/**
* Gets the size of this resource in bytes
*
* TODO 1.5: rename to lengthInBytes() or let it return some sort of size object
*
* #return The size of this resource in the number of bytes, or -1 if unknown
*/
long length();
So I think it would be ok if your IResourceStream implementation tells that the length is unknown and you stream the data directly as you get the records from the database.
You could return -1, indicating an unknown length, or you could write the result in a memory buffer or disk, before rendering it to the client.
You could also use this file as a cache, so that you don't need to regenerate it every time this resource is requested (remember you have to handle concurrent requests, though). Dedicated caching solutions (e.g. memcache, ehcache, etc.) can also be considered.
It may be cleaner than publishing a static file, although static files are probably better if performance is critical.
I ended up using an AbsractResourceStreamWriter subclass:
public class SitemapStreamWriter extends AbstractResourceStreamWriter
{
#Override
public void write(OutputStream output)
{
String HEAD = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
"<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\"\n" +
" xmlns:wicket=\"http://wicket.apache.org/dtds.data/wicket-xhtml1.4-strict.dtd\">\n";
try
{
output.write(HEAD.getBytes());
// write out a <loc> entry for each of my pages here
output.write("</urlset>\n".getBytes());
}
catch (IOException e)
{
throw new RuntimeException(e.getMessage(), e);
}
}
}
I was wondering if I can capture the result of an action after the result returns and the JSP is rendered. I want to be able to take the entire result (generated HTML) and push it into memcached so I can bring it via Nginx with-out hitting the application server. Any ideas?
PS: I know I can run the interceptor after the action executes but before the result returns and the JSP is rendered, but not after the JSP is rendered.
I haven't found a way to do this inside of struts2, your best bet it to create a servlet Filter and have it modify the OutputStream.
http://onjava.com/pub/a/onjava/2003/11/19/filters.html
Hey I know its quite late now to answer you might have already found out the answer, however for others to benefit I am posting the answer.
One thing that is very similar to what you are doing is done by sitemesh filter.
Yes, filter comes before and after the Struts2 filter itself, so you can mess with the inputs and outputs easily.
But struts does evaluate JSP/freemarker/velocity and generate the final html which is passed to the user. JSP is a bit trickey because internally a servlet is called but check out org.apache.struts2.views.freemarker.FreemarkerResult class, you can see the actual html getting generated in template.process(model, writer);. This writer is actually ServletActionContext.getResponse().getWriter();
Now to get the html all you need to do is
ServletActionContext.getResponse().getWriter().toString() //This does not work out of box. To get the toString() to work you need to use a ResponseWrapper - which is the same method to get result html in Filters. See- Programming Customized Requests and Responses.
Listing to modify resulting html in struts 2. This is not tested, but it is extracted from my code I have written earlier for custom template engine. I will probably post full description in Custom template engine for struts2
public class DecoratorInterceptor implements Interceptor {
public String intercept(ActionInvocation invocation) throws Exception {
final ActionContext context = invocation.getInvocationContext ();
HttpServletResponse responseParent = (HttpServletResponse)
context.get(ServletActionContext.HTTP_RESPONSE);
CharResponseWrapper wrapper = new CharResponseWrapper(responseParent);
ServletActionContext.setResponse(wrapper);
//Actual Action called
String result = invocation.invoke();
String htmlReturned = wrapper.toString();
//play with htmlReturned ...
String modifiedhtml = pushintoMemCache(htmlReturned );
CharArrayWriter car = new CharArrayWriter();
car.write(modifiedhtml );
PrintWriter out = responseParent.getWriter();
out.write(car.toString());
out.flush();
}
#Override
public void destroy() {
// TODO Auto-generated method stub
}
#Override
public void init() {
// TODO Auto-generated method stub
}
}
Read this article - http://struts.apache.org/2.0.6/docs/interceptors.html
SUMMARY:When you request a resource
that maps to an "action", the
framework invokes the Action object.
But, before the Action is executed,
the invocation can be intercepted by
another object. After the Action
executes, the invocation could be
intercepted again. Unsurprisingly, we
call these objects "Interceptors."
Question: How do you determine if the view has been generated? Do you set a request header or an some sort of a flag to determine if the view has been generated?
You could try throwing a MemCachedException to indicate that it is time to load into a mem cache. Your interceptor code could read
try {
return invocation.invoke();
} catch (MemCachedException mce) {
// Your code to upload to MemCache.
} finally {
// blah blah clean up.
}
Within your interceptor's intercept() method, the ActionInvocation parameter has a getResult() method which returns null before Action execution (i. e. before you call invocation.invoke() in your intercept() method) and contains an implementation of Result afterwards. That object should give you some way to access the data you need, but how this is done probably depends on the class that is actually used.
See also my somewhat related question and the answer I posted after figuring it out.