I'd like to verify a LL(1)-style parser using formal verification and TLA+.
But I don't have any idea how to model the specification of the parser.
Are there any examples about this,or can you give me some hint?
Related
I am considering to use the CUP parser generator for a project. In order to correctly parse some constructs of the language I am going to be compiling, I will need the lexer (generated by JFlex) to use information from the symbol table (not parse table -- I mean the table in which I will be storing information about identifiers) of the parser to generate the correct token type when its next_token() method is invoked. Since information in the symbol table depends statically on the program text, this will only work if the next_token() method is invoked "in lockstep" with the parser. In other words, this will work if the parser calls the lexer whenever it needs another token, but not if (for example) there is a parellel thread that is invoking the lexer and buffering tokens in a queue.
The question is thus: How does CUP call the lexer? Does it call it whenever it needs the next token? I could of course just write a CUP grammar specification and inspect the generated parser's source file to see what's going on, but that may be more work than necessary. I couldn't find any information on this on relevant websites.
Thanks a lot for any help you can offer!
I finished implementing my parser and scanner a while ago. Here's what I found:
CUP does indeed invoke the scanner as and when needed. It has always buffered one more token ahead of what has been recognized so far (the lookahead token). There is no fancy buffering of tokens ahead of time.
That being said, it can be tricky to set lexer states during parsing, as this can give rise to many grammar conflicts. I guess this is to do with the way CUP represents semantic actions embedded within productions. This forced me to abandon my initial design nonetheless, but not for the reason I was dreading.
Hope this helps someone!
Maybe this reply could be too late for you, but it could be useful for other users. The first thing to know is that a Parser couldn't do anything without a Scanner. As a matter of fact, the first parameter of the constructor of the parser is the scanner.
After the compilation of the .cup file, you will have, as output, a .java file that has the same name of the .cup one. Let's suppose its name is Parser.
So in the main class of your project you have to add the following lines:
TmpParser p = new TmpParser (new Scanner (new Reader (s)));
p.parse();
You should post this code into a try-catch block. With the method parse, the Parser starts its action and also it calls the next_token method of the Scanner, in order to recognize the token and verify if the grammar rules you wrote are right or not.
I don't know how late I'm to answer this question,
But I'm building 1 parser as a part of my course work..
I'm Using Lex and CUP for lexer and Parser, respectively. I'm also including my main class which calls parser which scans as in when required on get Token call
So My driver class will be :
// construct the lexer,
Yylex lexer = new Yylex(new FileReader(filename));
// create the parser
Parser parser = new Parser(lexer);
// and parse
Parser intern calls:
Parser.parse() {
...
this.cur_token = this.scan();
...
}
public Symbol scan() throws Exception {
Symbol sym = this.getScanner().next_token();
return sym != null ? sym : this.getSymbolFactory().newSymbol("END_OF_FILE", this.EOF_sym());
}
parser.parse();
Suppose I have this code:
(handler-case (read ...)
(parse-error (condition)
(format t "What text was I reading last to get this error? ~s~&"
(how-to-get-this-text? condition))))
I can only see the parse-namestring accessors, but it gives the message of the error, not the text it was parsing.
EDIT
In my case the problem is less generic, so an alternative solution not involving the entire string that failed to parse can be good too.
Imagine this example code I'm trying to parse:
prefix(perhaps (nested (symbolic)) expressions))suffix
In some cases I need to stop on "suffix" and in others, I need to continue, the suffix itself has no other meaning but just being an indicator of the action the parser should take next.
READ parses from a stream, not a string. The s-expression can be arbitrarily long. Should READ keep a string of what's been read?
What you might need is a special stream. In standard Common Lisp there is no mechanism for user defined streams. But in real life every implementation has such extensible streams. See for example 'gray streams'.
http://www.sbcl.org/1.0/manual/Gray-Streams.html
There's no standard function to do it. You might be able to brute-force something with read-from-string, but whatever you do, it will require some extra work.
I download an N-Triple file from dbpedia,but when I wanted to read it in to Jena model,some exceptions throw out.Below is a part of this file:
<http://dbpedia.org/resource/Jacky_Cheung>
<http://dbpedia.org/resource/Template:%E8%97%9D%E4%BA%BA> "\u9AD4\u91CD"#zh .
<http://dbpedia.org/resource/Jacky_Cheung> <http://dbpedia.org/resource/Template:%E8%97%9D%E4%BA%BA> "\u8EAB\u9AD8"#zh .
<http://dbpedia.org/resource/Jacky_Cheung> <http://dbpedia.org/resource/Template:%E8%97%9D%E4%BA%BA> "\u8840\u578B"#zh .
<http://dbpedia.org/resource/Jacky_Cheung> <http://dbpedia.org/resource/Template:%E8%97%9D%E4%BA%BA> "\u8A9E\u8A00"#zh .
The exception throws out is:
Exception in thread "main" com.hp.hpl.jena.shared.InvalidPropertyURIException: http://dbpedia.org/resource/Template:%E8%97%9D%E4%BA%BA
at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.splitTag(BaseXMLWriter.java:393)
at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.startElementTag(BaseXMLWriter.java:368)
at com.hp.hpl.jena.xmloutput.impl.Unparser$3.wTypeStart(Unparser.java:671)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wPropertyEltValueString(Unparser.java:488)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wPropertyEltValue(Unparser.java:473)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wPropertyElt(Unparser.java:339)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wPropertyEltStar(Unparser.java:811)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wTypedNodeOrDescriptionLong(Unparser.java:797)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wTypedNodeOrDescription(Unparser.java:727)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wDescription(Unparser.java:686)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wObj(Unparser.java:642)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wObjStar(Unparser.java:317)
at com.hp.hpl.jena.xmloutput.impl.Unparser.wRDF(Unparser.java:298)
at com.hp.hpl.jena.xmloutput.impl.Unparser.write(Unparser.java:200)
at com.hp.hpl.jena.xmloutput.impl.Abbreviated.writeBody(Abbreviated.java:143)
at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.writeXMLBody(BaseXMLWriter.java:500)
at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write(BaseXMLWriter.java:472)
at com.hp.hpl.jena.xmloutput.impl.Abbreviated.write(Abbreviated.java:128)
at com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write(BaseXMLWriter.java:458)
at com.hp.hpl.jena.rdf.model.impl.ModelCom.write(ModelCom.java:277)
at jena.ReadRDF.main(ReadRDF.java:45)
Java Result: 1
The problem is caused by "%E8%97%9D%E4%BA%BA",when use URIref.decode() to decode URI with this string,"%E8%97%9D%E4%BA%BA" represents tow Chinese characters.
But when I use Sesame to read this N-Triple file,it is OK without any problem.
My questions are that whether any way to solve this problem in Jena,and why dbpedia choose N-Triple to be the default RDF syntax?.It works bad with Non-ASCII languages.
Also ,I want to know that,if I want to publish my RDF data as Linked data,but the URIs of resources come with some Chinese and Japanese,should I decode the URIs at first?
Well, your question isn't completely clear because you asked about "reading in a Jena model" but the stacktrace you quoted actually starts with a call to the writer.
Jena, in general, tries very hard to conform to the relevant RDF recommendations from W3C and IETF. In particular, it tries to not generate any URI's which do not conform to the rules for valid URI's. This is compounded in the case of writing XML, because most RDF identifiers are not legal XML element ID's, meaning that you have to split the URI somewhere and use XML namespaces to make the full identifier. Not all RDF toolkits are as particular as Jena is about conforming to some of the rules in the standards.
Things you can try:
do you need to call Model.write() as part of your loading process? You should be able to load and process a model, without the check for legal URI's being invoked.
try writing the output using Turtle format, rather than XML. Turtle doesn't have the same restrictions as XML, and it's a heck of a lot easier for humans to read as well.
if there are particular ill-formed URI's in the data you are loading, look to see if there is a newer version of the data. Illegal URI's in dbpedia has been an issue in the past. If the illegal URI's are still there in the latest version, notify the dbpedia team about them.
try pre-processing your data to remove triples containing illegal URI's before they enter your processing chain.
As for URI's containing Chinese and Japanese characters, Jena conforms to the IRI spec, so as long as your URI's conform to that you should be OK.
I have to design a RESTful client and service in Java. For a GET request, i need to send a regEx pattern as a query parameter.
in the client i'm sending the pattern as
http://localhost:6520/restservice/foo?pattern=^BP$
i'm getting Illegal Endpoint address exception.
I even tried
http://localhost:6520/restservice/foo?pattern='^BP$'
and it still gives the same exception. Can anyone help me with this?
The ^ is an illegal character in URL's and the $ is a reserved character in URL's. You'd like to URL-encode those characters first. The correct URL would then end up to be http://localhost:6520/restservice/foo?pattern=%5eBP%24
Most server side programming languages and view technologies offer facilities to convert URL's that way. Since it's unclear which one you're using, we can't give any hints how to do this. You now at least know the right keywords to search on: "URL encoding". Google around together with the programming language as keyword.
I would like to parse standard security expressions like hasRole etc. to get their value in a variable (in a JSP). How can I do that? SpelExpressionParser is the standard EL parser, but I don't think it will parse the security expression.
The simpliest approach is to use a <sec:authorize> tag and modify a desired variable inside its body.
If you actually want to evaluate expressions manually, look at the source of AuthorizeTag - it gets the first bean of type WebSecurtyExpressionHandler from the web application context and uses it to obtain ExpressionParser and EvaluationContext.
I've posted an answer in this topic here - How to parse spring security expressions programmatically. I think this answers your question also.