I am trying to create a custom connector for Presto and InfluxDB in order to make it possible for Presto to run SQL queries on InfluxDB. Are there any examples of such a connector being available already?
Connectors are the source of all data for queries in Presto. Even if your data source doesn’t have underlying tables backing it, as long as you adapt your data source to the API expected by Presto, you can write queries against this data.
The only documentation that I found for writing a connector is:
https://prestodb.io/docs/current/develop/example-http.html
If anyone has other examples, can you please share it?
There are multiple connectors in the presto source tree.
When you're connecting to a data source having JDBC driver (probably not your case), extending presto-base-jdbc driver gives you almost all you need. See for example https://github.com/trinodb/trino/tree/master/presto-postgresql
When you're connecting to a non-JDBC-enabled data source (or you need more that it's possible with presto-base-jdbc), you need to implement all the relevant connector interfaces. There isn't good documentation for this other than Java interfaces & source code, but you can follow examples e.g. https://github.com/trinodb/trino/tree/master/presto-cassandra, https://github.com/trinodb/trino/tree/master/presto-accumulo
Yet another option is Greg Leclercq's suggestion to implement a Thrift connector. See his answer for directions.
Another option if you prefer to code in a programming language other than Java, is to implement a Thrift service and use the Thrift connector
I mainly want to use advantage to be able to access Fox tables larger than 2 gig. My programs are simple and are run from the command window.
I have Adv Data Archetect installed and have the ODBC driver installed.
I'm not very knowledgeable with connections, etc.
Can someone explain to me, provide a link or provide the code that I would need to be able to use and create 2 gig + tables.
Thanks
I cannot tell from the OP what you have actually done, but it sounds like you are expecting to be able to use an ODBC driver with an existing Visual FoxPro application without changing the application from the direct table access. That is not possible.
Here is a link to a screencast showing an example of using ODBC to get to a table that is over the 2GB limit. If I recall correctly, it shows how to use views to access the data; doing it that way can minimize the number of changes you need to make. More information about remote views can be found here.
You can also use ODBC "directly" with SQL pass through statements. It is also possible to use OLE DB with cursor adapters if you prefer that over ODBC.
I am coding in Delphi, using a TADOConnection to access ODBC compliant databases.
How do I copy a database leaving the new copy on the same database server?
And how do I rename? (I suppose I could copy & delete the original - if I knew how to copy).
ODBC does not provide for the copying or creating databases. That is a technology-specific (specific to the RDBMS) facility. The closest you can get is by creating and populating (copying) tables.
The only way you could do it would be to issue a db-specific command via an ODBC connection but for that we would have to know exactly what type of database you're using.
Are you using ODBC drivers, or ADO providers? If the later, you can look into the ADOX library, which provides vendor neutral support for working with the structure of databases. I don't know myself whether it supports operations on the entire database.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I'm starting to learn about COBOL. I have some experience writing programs that deal with SQL databases and I guess I'm confused how COBOL stores and retrieves data that is stored in a mainframe for example. I know that it's not like relational databases but every example program I've seen takes data straight from the command line and I know that's not how real world COBOL programs process the data. Can someone explain or show me a good resource that can explain it?
COBOL is just another third generation computer language. It is just a little older than most which doesn't mean it is somehow incomplete (actually it comes with quite a bit of baggage - but that is another story).
As with any other third generation language, COBOL manipulates data files in pretty much the same way that you would in a C program. Nothing odd, mysterious or magical about it. Files are opened, read, written and closed using the file I/O features of the language.
Various mechanisms are used to form a link between an actual file and the program. The details here are often specific to the operating system you are working under. Generally, COBOL implementations try to isolate themselves from the operating environment through a logical file name as opposed to an actual name. This added indirection is important when you are writing programs that will be ported to different platforms (e.g. write and test within an IDE on a Windows platform, and then run on a mainframe).
The following examples relate to an IBM Mainframe environment.
Within the IBM mainframe world, you will find that programs are run as either batch or on-line (e.g. CICS). I will not describe how to set up for file I/O under CICS (that's a long story). Programs that are used to manipulate files are usually batch. Here is a rough illustration of how a batch program works:
Batch programs are run via JCL. JCL is used to identify the program to run ('EXEC' statement) and to identify which files your program will reference using 'DD' statements. The function of a DD Statement is to form a logical connection between an actual file and a name your COBOL program will reference when it wants to refer to the file (this is the isolation mechanism mentioned earlier). For example,
JCLDDNAM DD DSN='HLQ.MY.FILE'...
would associate the 'DD' name 'JCLDDNAM' to the file named 'HLQ.MY.FILE'. This part is platform dependant so the details are specific to the operating environment.
In the 'FILE-CONTROL' section of your COBOL program, you connect the 'DD NAME' defined in your JCL with the name you will use on each I/O statement to reference that file. This connection is defined using the 'SELECT' statement.
For example,
SELECT MYFILE
ASSIGN JCLDDNAM
remainder of select
makes a connection between whatever file you associated with 'JCLDDNAM' in your 'JCL' to 'MYFILE' that you will later reference in COBOL I/O statements. The SELECT statement itself is part of the ISO COBOL standard. However, many COBOL implementations define some non-standard extentions to facilitate various quirks to their file subsystems.
Open, read, write, close files within the 'PROCEDURE DIVISION' of you program using the name 'MYFILE' as in:
OPEN MYFILE
READ MYFILE
CLOSE MYFILE
The above is highly simplified, and there are a multitude of ways to do this within COBOL. Understanding the complete picture will take some real effort, time and practice. The I/O statements illustrated above are part of the COBOL standard, but every vendor will have their own extentions.
IBM COBOL supports a wide range of file organizations and access methods. You can review the IBM Enterprise COBOL Language Reference manual here to get the syntax and rules for file manipulation, However, the User Guide provides a lot of good examples for reading/writing files (you will have to dig a bit—but it is all laid out for you).
The setup to reference an SQL database via a COBOL program is somewhat different but involves setting up a connection between your program and the database subsystem. Within the IBM world this is done throug JCL, other environments will use different mechanisms.
IBM COBOL uses a pre-processor or co-processor to integrate database access and data exchange. For example, the following code would retrieve some data from a DB2 database:
MOVE 1234 TO PERSON-ID
EXEC SQL
SELECT FIRST_NAME, LAST_NAME
INTO :FIRST-NAME, :LAST-NAME
FROM PERSON
WHERE PERSON_ID = :PERSON-ID
END-EXEC
DISPLAY PERSON-ID FIRST-NAME LAST-NAME
The stuff between EXEC SQL and END-EXEC is a pretty simple SQL select statement. The names preceded by colons are COBOL host variables used to pass data to DB2 or receive it back. If you have ever coded database access routines before this should look very familiar to you. This link provides a simple introduction to incorporating SQL statements into an IBM Enterpirse COBOL program.
By the way, IBM Enterprise COBOL is capable of working with XML documents too. Sorry for the heavy IBM slant, but that is the environment I am most familiar with.
Hope this gets you started in the right direction.
Who said you cannot use SQL to retrieve data from a cobol application, maybe without spending money?
A company I used to work for, did just that - with SQLite. This little gem of a public domain library compiles SQL statements to bytecode, then it executes them.
By replacing the "backend" level of SQLite with a custom interface to the C library that deals with Cobol files, it was possible to query the Cobol data from other languages, Python in that case. It worked -- within the limits of SQLite of course, but it was stable, it seemed relational enough and it didn't even require a DB server :-)
Traditional COBOL batch environments use a 'data section' of the cobol program to directly declare database connections, which are in turn set up in JCL. Since COBOL predates SQL, those would have tended to be various other types of databases, but it's likely that IBM made SQL work with DB/2. I imagine you'll get another answer from someone closer to this stuff. If you look at the SQL preprocessors available for use with other languages you'll get the idea -- a cursor becomes a native datatype and delivers query results to native variables.
Mainframe Cobol uses embedded SQL (kinda like SQLj), e.g.:
Procedure Division.
Exec SQL
Select col1, col2
from myTable
into :ws-col1, :ws-col2
where col0 = :col0
End-Exec
In this case, the host variables ws-col0, ws-col1 and ws-col2 are defined in the working-storage section. The database interface manages getting that data in the right place.
Very easy compared to the distributed stuff actually.
All the IBM mainframe shop's I've worked in have used COBOL which talked with a relational database. Generally that has been IBM's DB2. Please note that DB2 is a relational database that run's on mainframes. It can also be run on Windows or Linux.
Twenty years ago a predominate way to enter data into the DB2 mainframe database was to use CICS. CICS is "presentation level" software that comminicates with character based data entry screens. Consider CICS the functional equivelant of PHP or ASP.NET.
Today there are many more options to get data into DB2. CICS is still an option but your "presentation layer" could be PHP, ASP.NET, Win Forms, Java JSF, Powerbuilder. The key thing is that your development platform would need to be able to work with a DB2 database driver. The platform could be Windows, Linux, and possibly others.
My point is that data can get into the mainframe DB2 database in a many ways from many platforms. The COBOL language might be involved in data entry, reporting, altering data COBOL, etc. But it might only be part of an multiple tier application that could be part Windows, web, and mainframe . I could give specific examples if you have more information about the application you'll be working with at your internship.
In this OLE DB msdn example, (yes, it is dated 1997, but still works :)) I'm wondering what the historic point of OLE DB was if you still apparently wrote SQL to interact with the underlying datasource.
The one thing I'm thinking is if the ICommandText was not executed directly on the database, but instead somehow interpretted by OLE DB then passed off as a vendor specific SQL command/command to manipulate underlying datasource in format it understands. Is that true?
The intent of OLE DB technology is to have a generalized way to connect, work with transactions, and work with datasets. But you still have to write SQL queries. Remember, the intent of the SQL language was to have a generic language for querying data (but, as you've seen, certain parts of the language are platform-dependent). How you connect and how you query are mutually exclusive.
And, yes, the CommandText is executed directly in the database.
Perhaps you want something a level up such as NHibernate, Subsonic, Entity Framework, etc.?
I can vouch for the fact that there is NO translation done by OLE DB.
If you have different databases, you would have to provide different SQL statements based on the vendor. Always fun with dates and Oracle vs. MS databases...