Custom embedded language netbeans - parsing

Question

I'm developing plugin for Netbeans IDE that will provide support for new custom language. I have created parser and lexer for my custom language using ANTLR features. Besides, my language contains some "SQL-like" queries that are very complicated, so i decided to write separate grammar for "SQL-like" queries. Consequently, I had to make parser and lexer for my "SQL-like" language. As a result, I have two languages where "SQL-like" language is an embedded language.
Netbeans provides class EmbeddingProvider which is responsible for embedding languages. Here is my EmbeddingProvider:
#EmbeddingProvider.Registration(mimeType = "text/x-lorx", targetMimeType = "text/x-sqll")
public class LorxEmbeddingProvider extends EmbeddingProvider {
#Override
public List<Embedding> getEmbeddings(Snapshot snapshot) {
TokenHierarchy th = snapshot.getTokenHierarchy();
TokenSequence<LorxTokenId> ts = th.tokenSequence(LorxTokenId.getLanguage());
List<Embedding> embeddings = new ArrayList<>();
while(ts.moveNext()) {
Token currToken = ts.token();
if(currToken.id().ordinal() == LorxTokenType.SqllLiteral.id) {
embeddings.add(snapshot.create(currToken.text(), "text/x-sqll"));
}
}
return embeddings;
}
#Override
public int getPriority() {
return 140;
}
#Override
public void cancel() {
}
}
Annotation is used for determining top-level language("text/x-lorx") and it's embedded language ("text/x-sqll").
Method getEmbeddings(Snapshot snapshot) executes when we open some file in the editor or just move caret to another position. I use Snapshot class to get token sequence of the current opened file. In this code sample i am iterating tokens in search of SqllLiteral token(It's like [select * from ...]). If i find this token, i create new Embedding.
public class SqllParserFactory extends ParserFactory {
#Override
public Parser createParser(Collection<Snapshot> snapshots) {
return new SqllNBParser();
}
}
After finishing getEmbeddings(Snapshot snapshot) method, SqllParserFactory of an embedded language creates new parser for sqll language and then happens nothing. I would like to know if i'm on the right way and also i would be happy if someone gave me an advise how split embedded language text into tokens.

Answer 1

Instead of using EmbeddingProvider you can try to use LanguageProvider.
#ServiceProvider(service = LanguageProvider.class)
public class MyEmbeddingLanguageProvider extends LanguageProvider {
#Override
public LanguageEmbedding<?> findLanguageEmbedding(Token<?> token,
LanguagePath languagePath, InputAttributes inputAttributes) {
Language embeddedLanguage = MimeLookup.getLookup("text/sqll").lookup(Language.class);
if (embeddedLanguage != null && languagePath.mimePath().equals("text/x-lorx")) {
if (token.id().ordinal() == LorxTokenType.SqllLiteral.id) {
return LanguageEmbedding.create(embeddedLanguage, 0, 0, true);
}
}
return null;
}
#Override
public Language<?> findLanguage(String mimeType) {
return null;
}
}
It should work for tokenizing both languages (which can result in nice syntax coloring). Unfortunately parsing of embedded language may not work.

Answer 2

The problem has been solved by overriding method in class LanguageHierarchy. I was trying to override this method before, but I used it incorrectly. The problem was in passing wrong params to method LanguageEmbedding.create(SqllTokenId.getLanguage(), 0, 0);. Instead of passing zeros, I passed length of my token, so it was skipping my token, because the second param is startSkipLength and the third is endSkipLength. The code below is correct. The LorxTokenType.SqllLiteral is token of my "sql-like" query. It is resolved as an embedded language now and processing by another lexer and parser (in my case by Sqll lexer and parser).
#Override
protected LanguageEmbedding<?> embedding(Token<LorxTokenId> token, LanguagePath languagePath, InputAttributes inputAttributes) {
if(token.id().ordinal() == LorxTokenType.SqllLiteral.id) {
return LanguageEmbedding.create(SqllTokenId.getLanguage(), 0, 0);
}
return null;
}