Most of my code is using binary representation for utf8 strings. Unfortunately the scanner and parser, that I generated with Leex and Yecc respectively, don't seem to accept binaries. Is there any option or anything to do for Yecc and Leex generated code to 1) accept binaries as input and 2) produce binaries?
Related
Does anyone know of any tool(s) that can convert ANTLR v4 grammar files (.g4 extension) to tree-sitter grammar files (.js extension)? It would also be fine if I had to chain a couple conversion tools together. For example, going from foo.g4 (antlr4) to foo.ebnf (intermediary format) to foo.js (tree-sitter). Thank you!
I tried using this tool to go from g4 to ebnf, and then this tool to go from ebnf to tree-sitter js, but to no avail. The first tool seemed to create some junk at the bottom of the file which gave the second tool trouble. Additionally, the second tool seems to expect each definition to be completely on one line (and the first tool breaks each definition up into multiple lines for readability).
Are there any Isabelle/Isar languages parsers available? Specifically one that will parse the syntax and output something useful, ideally in a language-agnostic format such as JSON. I would like to basically have a JSON representation of an Isabelle/Isar .thy file.
Is there an existing parser to parse json-ld to markdown? I want to generate documentation from my jsonld file. If such a thing doesn't exist, how should I go ahead writing one? or perhaps I could use a json to markdown converter? Any suggestions on how could do this?
I was just googling for such a program, and found your question.
The closest things I could find are: ocxmd, which is an extension to Markdown; and md-ld, which does not even use proper Markdown - instead, it apparently creates an incompatible version of the format which can be parsed to JSON-LD.
If I were writing such a converter in Python, I would use:
pyld to parse JSON-LD files and expand them using the #context;
And a template engine, likely Jinja2, to generate Markdown representation of every node of the JSON-LD document.
The program would be based on recursion. You might have separate functions to display:
URIs,
Numbers,
Images,
...
The program will recurse over the JSON-LD document and convert each of its sections into Markdown format.
I want to analysis OCaml files (.ml) using OCaml. I want to break the files into Abstract Syntax Trees for analysis. I have attempted to use camlp4 but have had no luck. Has anyone else successfully done this before? Is this the best way to parse an OCaml file?
(I assume you know basic parts of OCaml already: how to write OCaml code, how to link modules and libraries, how to write build scripts and so on. If you do not, learn them first.)
The best way is to use the genuine OCaml code parser used in OCaml compiler itself, since it is 100% compatible by definition.
CamlP4 also implements OCaml parser but it is slightly incompatible with the genuine parser and the parse tree is somewhat specialized for writing syntax extensions: not very good for any other kind of analysis.
You may want to parse .ml files with syntax extensions using P4. Even in this case, you should stick to the genuine parser: you can desugar the source code by P4 then send the result to your analyzer with the genuine parser.
To use OCaml compiler's parser, the easiest approach is to use compiler-libs.common OCamlFind package. It contains the parser and type checker of OCaml compiler.
Start from modifying driver/compile.ml of OCaml compiler source, it implements the major compilation phases: calling preprocessor, parse, typing then code generation. To parse .ml files you should modify (or simplify) Compile.implementation. For .mli files Compile.interface.
Good luck.
Couldn't you use the -dparsetree option to the ocaml compiler?
hello.ml:
let _ = print_endline "Hello AST"
Now compile it:
$ ocamlc -dparsetree hello.ml
Which results in:
[
structure_item (hello.ml[1,0+0]..[1,0+33])
Pstr_eval
expression (hello.ml[1,0+8]..[1,0+33])
Pexp_apply
expression (hello.ml[1,0+8]..[1,0+21])
Pexp_ident "print_endline" (hello.ml[1,0+8]..[1,0+21])
[
<label> ""
expression (hello.ml[1,0+22]..[1,0+33])
Pexp_constant Const_string("Hello AST",None)
]
]
See also this blog post on -ppx extensions which has some info on extension point syntax extensions (the new way of writing syntax extensions in OCaml 4.02). There is info there on various AST manipulation modules.
I am starting a class project that regards adding some functionality to Go.
However, I am thoroughly confused on the structure of Go. I was under the impression that Go used flex and bison but I can't find anything familiar in the Go source code.
On the other hand, the directory go/src/pkg/go has folders with familiar names (ast, token, parser, etc.) but all they contain are .go files. I'm confused!
My request is, of anyone familiar with Go, can you give me an overview of how Go is lexed, parsed, etc. and where to find the files to edit the grammar and whatnot?
The directory structure:
src/cmd/5* ARM
src/cmd/6* amd64 (x86-64)
src/cmd/8* i386 (x86-32)
src/cmd/cc C compiler (common part)
src/cmd/gc Go compiler (common part)
src/cmd/ld Linker (common part)
src/cmd/6c C compiler (amd64-specific part)
src/cmd/6g Go compiler (amd64-specific part)
src/cmd/6l Linker (amd64-specific part)
Lexer is written in pure C (no flex). Grammar is written in Bison:
src/cmd/gc/lex.c
src/cmd/gc/go.y
Many directories under src/cmd contain a doc.go file with short description of the directory's contents.
If you are planning to modify the grammar, it should be noted that the Bison grammar sometimes does not distinguish between expressions and types.
lex.c
go.y
The Go compilers are written in c, which is why you need flex and bison. The Go package for parsing is not used. If you wanted to write a self hosting compiler in Go, you could use the Go parsing package.