JSqlParser for Data Lineage? - jsqlparser

Question

We are a data warehouse development team and most of our ETL logic can be expressed as a series of SQL select statements. I am looking for a tool to extract data lineage in a structured manner by parsing the queries.
The query and the simplified lineage output would look like this:
Query:
SELECT A AS COLUMN_1, B AS COLUMN_2, A+B AS COLUMN_SUM FROM MYTABLE;
Output
COLUMN_1: MYTABLE.A
COLUMN_2: MYTABLE.B
COLUMN_3: MYTABLE.A
COLUMN_3: MYTABLE.B
Is JSQLParser a good tool for this purpose? Any pointers or experiences on how to use the tool would be appreciated too.

Answer 1

JSqlParser does the parsing and gives you a structured way to look at your SQL. By the way JSqlParser is pretty good at this.
But it has no knowledge about your database schema, therefore it cannot know if column_1 A is from table MYTABLE. A more obvious example would be
select a, b from table1, table2
This knowledge must somehow injected by you :).
To make a simple parse do something like
Statement statement = CCJSqlParserUtil.parse(sql);
To extract used columns you could use the TablesNamesFinder utility, provided by JSqlParser like
Select selectStatement = (Select) statement;
TablesNamesFinder tablesNamesFinder = new TablesNamesFinder() {
#Override
public void visit(Column tableColumn) {
System.out.println(tableColumn);
}
};
System.out.println(" and tables=" + tablesNamesFinder.getTableList(selectStatement));
As you see one way to surf your data is some kind of visitor pattern.
If you have more questions feel free to use JSqlParsers gitter room or file an issue at github.