I'm trying to work out how these two pieces of the jigsaw interact and fit together when connecting to an MS-SQL server on linux.
As I understand it, FreeTDS is protocol (i.e. a set of rules) for talking to MS-SQL and it is the thing that actually does the talking. Unixodbc is a driver that implements the ODBC API, i.e. implements a set of functions I guess.
Why are both things necessary? Can anyone elaborate on my sketchy understanding of what these two things actually do?
unixODBC is a 'DriverManager' for ODBC. You can use unixODBC when on a Linux or *nix system to connect to any ODBC-capable database. Doing so means that you can write one lot of database queries which you should be able to use between different databases. If you were not on Unix, you would use a different Driver Manager, for example the built-in MS Office one.
To make all the components clear: if you're using a language, let's say Python, to connect to SQL Server, your connection might pass from Python's pyodbc (translates python objects to and from unixODBC), to unixODBC (manages drivers, such as FreeTDS), to FreeTDS (translates unixODBC objects to and from the TDS protocol, which Microsoft embraces) to SQL Server.
The unixODBC website http://www.unixodbc.org/ says:
An ODBC application makes ODBC calls to the DriverManager. The DriverManager carries out a number of tasks for the application such as:
ensuring the proper driver is loaded/unloaded
validation tasks
3.5 to 3.0 to 2.0 call and data mapping
Most calls to the DriverManager get passed onto the loaded Driver to be further processed but that is of little concern to the application.
Some advantages to using an ODBC DriverManager include:
portable data access code
runtime binding to a Data Source
ability to easily change the Data Source
Briefly, it is the Driver Manager which reads your DSN, looks at the configured data sources, and decides where and how to connect.
Depending on which database you use, you will need a different driver. This piece of code 'translates' your requests made using ODBC to the right protocol for the relevant database management system. This is the component that would need to be different for different data sources. In your case, TDS is the protocol used by MS SQL Server. FreeTDS is a free software implementation of this protocol.
See also Wikipedia https://en.wikipedia.org/wiki/Open_Database_Connectivity (emphasis kept):
ODBC accomplishes DBMS independence by using an ODBC driver as a translation layer between the application and the DBMS. The application uses ODBC functions through an ODBC driver manager with which it is linked, and the driver passes the query to the DBMS. An ODBC driver can be thought of as analogous to a printer driver or other driver, providing a standard set of functions for the application to use, and implementing DBMS-specific functionality. An application that can use ODBC is referred to as "ODBC-compliant". Any ODBC-compliant application can access any DBMS for which a driver is installed. Drivers exist for all major DBMSs, many other data sources like address book systems and Microsoft Excel, and even for text or CSV files.
FreeTDS is the implementation of the TDS protocol, it handles everything and is fully capable of operating without unixODBC.
ODBC is a wrapper around FreeTDS and provides a common API that is better documented than FreeTDS' internal API.
As FreeTDS.org/faq.html indicates:
FreeTDS offers three client libraries and one internal one (libtds). We generally encourage people to use one of the client libraries, and discourage writing to libtds, because the latter is evolving, more subject to change, less well documented, and harder to use.
Both pieces aren't necessary, it's just that some people prefer to use wrappers rather than learn a new API to do something similar to what they already know in a different API. As FreeTDS' FAQ indicates, they support other open APIs other than ODBC and their internal library is fully capable of handling the connection all on its own.
Related
How does AppDynamics and similar problems retrieve data from apps ? I read somewhere here on SO that it is based on bytecode injection, but is there some official or reliable source to this information ?
Data retrieval by APM tools is done in several ways, each one with its pros and cons
Bytecode injection (for both Java and .NET) is one technique, which is somewhat intrusive but allows you to get data from places the application owner (or even 3rd party frameworks) did not intend to allow.
Native function interception is similar to bytecode injection, but allows you to intercept unmanaged code
Application plugins - some applications (e.g. Apache, IIS) give access to monitoring and application information via a well-documented APIs and plugin architecture
Network sniffing allows you to see all the communication to/from the monitored machine
OS specific un/documented APIs - just like application plugins, but for the Windows/*nix
Disclaimer : I work for Correlsense, provider of APM software SharePath, which uses all of the above methods to give you complete end-to-end transaction visibility.
I am a new commer in SOC, I have a question about two systems,
OpenWRT and μTenux.
It seems both solutions can work, but what is the difference between each?
When should we use OpenWRT for the basis of a new application? When should we use μTenux?
I want to write a SOC application, which can make socket connection to website and fetch a config file. Which system should I choose?
OpenWrt is a Linux based OS tailored towards router software, i.e. it has web front-end for configuring various network parameters. It is more suitable for server side.
μTenux seems to be bare metal real-time OS, i.e. not Linux based. It is capable of fetching a config file via web.
If you just want to write a client application I would recommend Buildroot.
What is the best way to write a client-server application under delphi? I know there's a DataSnap technology, but it's not in Professional version. Do You have any experience that You can share?
This is fairly wide open question, as it can depend on your database decision.
DataSnap really allows for N-Tier solutions, if your looking for Client Server you have
most everything you need in the professional version depending on the Database Choice.
For Client Server:
Client Server Architecture is when the Client communicates directly with the server.
There are several frameworks available they all follow the same pattern.
DB Connection -> Query -> (Optional Provider -> TClientDataset) -> TDataSource -> Visual Control
DBX
TSqlConnection - Connects to the Database
TSqlQuery - Query against DB producing uni-directional Dataset
TSqlStoredProc - Executes Stores Procedures against DB
ADO
TAdoConnection - Connects to Database
TAdoQuery - Query against DB producing Bi-Directional Dataset
Common Components
TClientDataSet - In Memory dataset that is bi-directional
TDatasetProvider - Takes other datasets and ties the data to TClientDataset
TDataSource - Ties a Dataset to a data-aware visual control
There are several other options available depending on Database Choice.
However, you seem to be asking about N-Tier (Middle-Tier) type solutions
For N-Tier
N-Tier architecture is when the Client communicates with Middle Tier that then communicates with the Server. It's referred to N-Tier as you have option to have multiple Middle Tiers or Application Servers.
Commercial Options (Required additional $$ to be spent)
DataSnap
DataAbstract
RemObjects SDK (Part of DataAbstract but can be used by itself)
KBMMw
Midware
I personally don't know of any free or open source options, although I suspect some exist.
Two options:
DIY (Do It Yourself). Write a communications layer and protocol yourself using Indy and/or ICS internet components. A lot of hard work and needs a lot of testing to get right.
Use a ready made framework such as kbmMW: http://components4developers.com/ or RemObjects: http://www.remobjects.com/ Both are not free but well worth the money you pay even if only measured by the development time/costs that you spare.
You can use
WST is a free and open source toolkit for web services consumption and creation with support for SOAP and XmlRPC and JsonRPC (the JsonRPC support is available only for FPC). It is compatible with Delphi. Better check out from svn as the 0.5 release is actualy outdated.
With Delphi Professional it is possible to write simple (no WS-* standards, no Soap 1.2 servers) SOAP client and server applications.
In many cases, Soap offers advantages regarding cross-platform / cross-language integration, standardization, design-by-contract and mature implementation guidelines, best practices and patterns.
For Soap there are great (and free) tools like SoapUI and IDE editors for Web Service Description Language (WSDL) documents like NetBeans.
Take a look at our Open Source Client/Server ORM.
It's multi-tier compatible, and you can have ORM at both Client and Server level.
ORM is used everywhere, and JSON is the format chosen for the Client/Server transmission.
You can start your application as local application, then just by changing the class type used to access to the data, it will become a Client/Server application communicating via Named Pipes, HTTP/1.1 or GDI messages.
It was designed to work with SQLite3 as a small but efficient database engine on the server side, but you can use the ORM without SQlite3. There is a pure Delphi in-memory engine provided, if you prefer.
This framework try to implement N-Tier architecture from the bottom up.
The upcoming 1.13 version will have a powerful filtering and validation mechanism, perfect for N-Tier architecture. There is some User-Interface units, with full reporting (and pdf generation), able to create most of the User Interface from code, using the ORM layout of the data.
It's based on the RESTful paradigm to access the data from the Client, via JSON. And there is a easy way of implementing Client/Server Services if the RESTful approach is not enough, just like DataSnap.
It's Unicode ready (uses UTF-8 at all internal level), and works with every version of the IDE, from Delphi 6 up to XE (even the Starter edition).
Since a few months ago I stopped to implement new projects with this kind of architecture (n-tiers, 2-tiers) Based on Delphis and specific DB technologies. I believe these architecture are not future prof. The architecture i'm using now is a 2-pier one. The server is a normal HTTP server. It works as app server* and optionally provides a web client. Developing clients in Delphi it's harder but worth it. Since specif tools are not available as the ones offered for DB connections, I use indy to send and receive data from the HTTP server. I do a GET request to fetch data and then parse it to show it on the GUI. Then a POST request to update or insert new data. The HTTP server handle all business logic :-)
Apart of being future prof, this architecture is cheaper and platform independent. And if you analyze it, this is the same architecture used by most mobile apps. So, if you plan to write a mobile client in the future, consider developing the app server with script languages (Python, PHP, Ruby, etc.).
That's my recommendation. Don't forget: Great things require great commitments!
An App Server is a service which provides your application (thin client) with with an interface to get and send data. Also it control the business logic. Your application don't care about DB's or controlling record relations and data constrains. That's all is transparently done by the app server.
For general-purpose client-server communication you can use our lightweight MsgConnect product. This is a cross-platform MOM (message-oriented-middleware).
I'm a C++ programmer, but my question is about the Delphi network socket library Indy.
Essentially as Microsoft crippled raw sockets with WinXP SP2 so that TCP data cannot be sent over them, does this then also apply to Indy? One of my friends recommended it to me rather then using the WinSock API.
If not, then is the only way to use raw TCP sockets using the WinPCap API?
If it doesn't, I know that synapse for Delphi does. The latest version now supports D2009 and it is being actively supported.
As you can see from the other answers there are a lot of Delphi libraries to simplify network programming. However, AFAIK they all provide a friendlier layer over the WinSock API (on Windows, over the standard socket API on other OSs) and thus are not able to do anything that the WinSock API isn't able to do. In particular they may or may not support raw socket access, but if the OS fails to send data over the socket the libraries will only return errors as well.
For more information about the limitations of raw sockets on most recent Windows OS versions see this MSDN page, in particular the section "Limitations on Raw Sockets". It states quite clearly that
To get around these issues, it may be required to write a Windows network protocol driver (device driver) for the specific network protocol.
Neither of the Delphi network programming libraries comes with its own network protocol driver AFAIK. And neither will help you with permission issues on limited user accounts, or the interference of the Windows Firewall.
As you mention WinPcap in your question you know it is capable of doing what you want. There are Delphi wrappers for it, like Magenta Systems Internet Packet Monitoring Components. However, as a C++ programmer you may be better off to use the library directly from C++, as Delphi adds nothing in regard to the socket programming you want to do. In particular you won't need the extra layer to hide platform differences that some of the Delphi libraries provide, as you will be programming directly against the libpcap API.
Indy works quite well. I use it quite a bit with few problems.
If your looking for more basic socket interaction you could use these other components:
TTCPClient
TCPServer
TClientSocket
TServerSocket
All of these components also ship with Delphi.
Don't know about winpcap directly, but I believe there is a delphi wrapper for it. I don't currently know where I've seen it but it shouldn't be hard to find using Google code search.
you should check ICS components
ICS is a free internet component
library for all Delphi, C++Builder,
BDS and RAD Studio versions (win32).
It includes TCP, UDP, raw sockets,
clients, servers, as well as many high
level protocols such as FTP, SMTP,
POP3, NNTP, HTTP and more.
http://www.overbyte.be/
I would rather prefer synapse over than Indy, it's architecture is more simpler thus easier to understand.
I am thinking about using SQLite
It is a self-contained, server less, zero-configuration, transactional SQL database engine and is open source.
Will I gain anything by using Blackfish instead of SQLight ?
Why not Firebird?
http://www.firebirdsql.org
http://www.firebirdfaq.org
"...Firebird is a relational database offering many ANSI SQL standard features that runs on Linux, Windows, and a variety of Unix platforms. Firebird offers excellent concurrency, high performance, and powerful language support for stored procedures and triggers. It has been used in production systems, under a variety of names, since 1981...."
Stick with FOSS (Free Open Source Software).
Both SQLite and Firebird are excellent choices. Both fill your requirements. Both are very reliable, zero-configuration and support transactions.
Without knowing enough about your intended use, Firebird would be my first choice because it makes possible to migrate to a Client Server deployment with close to zero effort and it has a very robust set of features. It is all about the options.
IMHO Blackfish is not a god choice - "Blackfish SQL runs on both the .NET framework and on the Java platform." - enough said.
Use SQLite.
Much smaller, less overhead, no licensing issues etc.
Laslty, only you can decide which one will do everything you need a db system to do. Which one has all of the features you need to support?
John
I will vote for Sqlite first, because it's compact, lite and fast, but depend on your application you may go with other choices like Firebird or PostgreSql.
for example Sqlite has limitation to one writer at the same time, it may not be problem for most of desktop applications that used by one user, but it will not scale for more uses in future.
you can go with FireBird embedded as solution which will be act like Sqlite, then move to firebird server when you need more users.
In other side BlackFish database, will force you to install .Net framework on your client's machines, which is something isn't good IMO, specially if you need to distribute your application on Internet, beside it will require license when it scale for more than developer edition.
Another thing to consider is how are you going to access your data. If you are using DBX4 for instance, it is very easy to change what database you are looking at. In which case I would suggest trying both (plus any others that people highly recommend you try). On the other hand if you are using the the Interbase or ADO data access components then your ability to change is somewhat more limited.