Invalid object name '#Results' SSIS when using sp in dataflow - stored-procedures

I have a stored procedure that runs fine in SQL Management Studio but I'm having problems running it in SSIS 2008 R2. If I run it as an Execute SQL Task, it runs fine without any errors but when I use it as an ADO NET Source in a Data Flow Task, I get an error messaging
Invalid object name #Results (Microsoft SQL Server, Error:208)
However when I click Preview, I do get rows of data displayed.
I don't have access rights to modify the stored procedure so I'm not sure what is going on inside the stored procedure itself but as I have said previously, I can run the stored procedure in management studio and when used in an Execute SQL Task in SSIS.

One of the steps in SSIS is validation of metadata - the contract says we should have an integer and then a character size 8. When the data flow database source components (ado or ole) attempt to get their metadata, it's basically boils down the first query that is found.
The approach here is the same hack we use with dynamic tables in stored procedures. Change the stored procedure, which you've specified you cannot do, to provide a hint to SSIS on the expected metadata.
CREATE PROCEDURE dbo.Sample
AS
BEGIN
SET NOCOUNT ON;
-- Any condition that will never evaluate to true
IF NULL = NULL
BEGIN
-- SSIS will key off of this query even
-- though it is impossible for this branch to ever execute
-- So, define our metadata here
SELECT
CAST(NULL AS int) AS MyFirstColumn
, CAST(NULL as char(8)) AS SomeCodeColumn;
END
-- Assume complex queries here that banjax the metadata
-- yet ultimately return the actual data
SELECT TOP 1000
CAST(ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS int) AS MyFirstColumn
, CAST(LEFT(AC.name, 8) AS char(8)) AS SomeCodeColumn
INTO
#RubeG
FROM
sys.all_columns AS AC;
SELECT
RG.MyFirstColumn
, RG.SomeCodeColumn
FROM
#RubeG AS RG;
END
For sources of SQL Server 2012+, you can try to specify the WITH RESULT SETS property to your EXECUTE call.
EXECUTE dbo.Sample
WITH RESULT SETS
(
(
c1 bigint
, c2 varchar(8)
)
);
Biml
Sample biml package definition.
download and install BIDS Helper
Open/create Integration Services project type
Add new biml file
Paste following definition
Adjust connection string value in line 5 (for OLE) an 8 (for ADO.NET)
Ensure the stored procedure dbo.Sample exists
Remove DFT Sample Result Set if using a 2008 database
Code here
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Connections>
<Connection
Name="tempdb"
ConnectionString="Data Source=.\dev2014;Initial Catalog=tempdb;Provider=SQLNCLI10.1;Integrated Security=SSPI;"
/>
<AdoNetConnection
Name="CM_ADO"
ConnectionString="Data Source=localhost\dev2014;Integrated Security=SSPI;Connect Timeout=30;Database=tempdb;"
Provider="SQL"
/>
</Connections>
<Packages>
<Package Name="so_31206473">
<Tasks>
<Dataflow Name="DFT Sample">
<Transformations>
<OleDbSource ConnectionName="tempdb" Name="OLESRC dbo_Source">
<DirectInput>EXECUTE dbo.Sample</DirectInput>
</OleDbSource>
<DerivedColumns Name="DER Placeholder" />
</Transformations>
</Dataflow>
<Dataflow Name="DFT Sample RESULTS SET">
<Transformations>
<OleDbSource ConnectionName="tempdb" Name="OLESRC dbo_Source RS">
<DirectInput>
<![CDATA[EXECUTE dbo.Sample
WITH RESULT SETS
(
(
c1 bigint
, c2 varchar(8)
)
);]]>
</DirectInput>
</OleDbSource>
<DerivedColumns Name="DER Placeholder" />
</Transformations>
</Dataflow>
<Dataflow Name="DFT SampleADO">
<Transformations>
<AdoNetSource ConnectionName="CM_ADO" Name="ADOSRC dbo_Sample">
<DirectInput>EXECUTE dbo.Sample</DirectInput>
</AdoNetSource>
<DerivedColumns Name="DER Placeholder" />
</Transformations>
</Dataflow>
<Dataflow Name="DFT SampleADO RESULTS SET">
<Transformations>
<AdoNetSource ConnectionName="CM_ADO" Name="ADOSRC dbo_Sample">
<DirectInput>
<![CDATA[EXECUTE dbo.Sample
WITH RESULT SETS
(
(
c1 bigint
, c2 varchar(8)
)
);]]>
</DirectInput>
</AdoNetSource>
<DerivedColumns Name="DER Placeholder" />
</Transformations>
</Dataflow>
</Tasks>
</Package>
</Packages>
</Biml>
Sample metadata for an OLE Source
WITH RESULTS SET metadata for an OLE Source
The results are the same for ADO.NET providers, I simply didn't notice that nuance to the question when I built my screenshots. Updated Biml makes it trivial to add those in though.

Related

Java Web with Weblogic, DB Informix stored procedure data — impossible problem

I have Java Web on Weblogic; database is Informix.
The process is as follows:
User query data.
Create serial(only).
Using stored procedure with serial.
SP content like:
insert reporttable
select data from table1
insert reporttable
select data from table2
if(reporttable.count==0)
insert reporttable select 'NO DATA'
Query reporttable with serial.
Show on Web.
Important problem:
table1 has data count 10(data1,data2.......data10)
reporttable result data count 3(data1, data2, NO DATA) impossible
Important!!! The implementation does not process any exceptions.
When the problem occurs, any query on the data shows the same problem.
But when I restart Weblogic (using the same parameters), the query has no problem.
I have no idea to solve the problem; can you help?
I find the error reason.
Test : rename table name
sp use table1、table2、table3
unknown reason maybe abnormal connection
java.sql.SQLSyntaxErrorException: [FMWGEN][Informix JDBC Driver][Informix]The
specified table (table1) is not in the database.
Error message only trigger on first
Execute sp again,no error, and executeing neglect table1
weblogic restart jndi connection
Execute sp result normal

Howto log to 2 instances of same type of sink (Seq)?

Possible?
Cannot find a "sink forwarder", where one sink can forward to several other sinks, possibly of the same type.
Serilogs documentation (https://github.com/serilog/serilog/wiki/AppSettings)
clearly states that
NOTE: When using serilog: keys need to be unique.*
so adding the same Seq sink several times doesnt seem to be a good idea.
I'm looking for the same concept as in log4net, where one logger can hold several appenders.
Unfortunately the <appSettings> config provider for Serilog doesn't support this case; the appSettings.json one does, if you're able to use it, otherwise configuring the sinks in code WriteTo.Seq(...).WriteTo.Seq(...) is the way to go.
Semi-workaround style of solution:
Put a "read these keys" in appsettigs
Example 1: Read one key
<add key="SerilogToHttpKeys" value="MyMachineA" />
Example 2 (which solves the problem): Read many keys
<add key="SerilogToHttpKeys" value="MyMachineA, MyLocalMachine, MachineOnTheMoon" />
Both cases "points" to an unlimited number of keys, that are then read via code (see 2) and hence be tweaked w/out recompiling
<add key="MyLocalMachine" value="http://localhost:5341/;juzOPqqqqqqqq" />
<add key="MyMachineA" value="http://10.107.14.57:5341/;m8QVnDaqqqqqqqqqqqqq" />
<add key="MachineOnTheMoon" value="http://10.107.14.62:5341/;Ah0tSzqqqqqqqqqqqq"
Loop the keys in code - each key points to a http address with an API key, which is used for logging to Seq, but change the structure of each entry, and you could log to file ect.
foreach (var aKey in System.Configuration.ConfigurationManager.AppSettings.Get("SerilogToHttpKeys")
.Split(',')//Use , as separator
.Select(s => s.Trim()))
{
var fields = System.Configuration.ConfigurationManager.AppSettings.Get(aKey);
var separator = ';';
string serverUrl = fields.Split(separator)[0];
string apiKey = fields.Split(separator)[1];
loggerConfiguration = loggerConfiguration.WriteTo.Seq(serverUrl: serverUrl, apiKey: apiKey);
}
I use this for logging both to my server and my dev machine at the same time - its easier to keep the localhost Seq open when errors occour, and see if I can find them there instead of logging into the server. However, in case my devmachine is not online, I have the logs on the server as well. Off course, if more than one persons accesses the server licenses are needed for Seq, but in a simple "one dev, one dev-machine, one server" it works.

Creating database in InterBase

I try to create a InterBase database on Delphi using TIBDatabase type.
ibdb:=TIBDatabase.Create(nil);
with ibdb do
begin
SQLDialect:=3;
DatabaseName:=Self.name;
Params.Clear;
Params.Add('USER "SYSDBA"');
Params.Add('PASSWORD "masterkey"');
Params.Add('PAGE_SIZE 4096');
LoginPrompt:=false;
try
CreateDatabase;
except on E: Exception do
ShowMessage('Can''t create database. '+E.Message);
end;
end;
Self.name is string like a 'localhost:C:\db.ib'.
Code works without errors. Database file created. But IBConsole doesn't show this database on IB server. How can I check that database was added on server?
The following works for me and avoids all the problems you've mentioned so far:
1 Start a new Delphi project
2 Drop a TIBDatabase onto the form
3 Add the following CreateDatabase procedure to the form
procedure TForm1.CreateDatabase;
begin
IBDatabase1.SQLDialect:=3;
IBDatabase1.DatabaseName := 'D:\aaad7\interbase\newdb.gdb'; //edit for
// your system & check that you have file-creation rights in the
// database folder you want to use.
IBDatabase1.Params.Clear;
IBDatabase1.Params.Add('USER ''SYSDBA'''); // Note that this line
// contains 5 single-quotes and no double quotes
// Also note that the omission of the usual '=' between the
// parameter name and value for the USER and PASSWORD is deliberate,
// because it's evidently required by the IB api
IBDatabase1.Params.Add('PASSWORD ''masterkey'''); // Ditto
IBDatabase1.LoginPrompt:=false;
try
IBDatabase1.CreateDatabase;
except on E: Exception do
ShowMessage('Can''t create database. '+E.Message);
end;
end;
4 Add a TButton and set its OnClick event to call CreateDatabase.
5 Compile & run and click Button1.
Using Delphi 7 and Interbase XE7, this code successfully creates the newdb.gdb database and this database stays on disk after the Delphi application terminates. I can then open the database in IBConsole using the procedure described below. I cannot delete the databse file via the windows shell while IBConsole is running but I can after I close it.
Do you get the same result and, if you do, does that solve your problem and if not, why not?
Btw, I've written the CreateDatabase code to avoid the use of "with", because it usually creates more problems than it avoids and I loathe it, and
to avoid the dynamic creation of the TIBDatabase component, because that serves no useful purpose.
Notes
Once you have created a database in code as above, there is no need to "attach" it to IBConsole.Exe or to the Interbase server in order to be able to use it.
You could set up an IBDatabase, IBTransaction and IBQuery to connect to it, and create tables in it by issuing "CREATE TABLE ..." SQL statements to it from the IBQuery.
If you want the database to be listed in the IBConsole utility, you can either
a) Add it manually using the procedure in Previous Answer below or
b) Add it in code to the IBConsole configuration file, IBConsole.XML, which you will find in C:\users[your user name]\Appdata\Roaming\Embarcadero\Interbase, using a Delphi XML access library of your choice. If you add it using the manual procedure below, then close IBConsole.Exe so that the config file is updated on disk, you'll be able to examine the XML file and find the node for the database under the Server/Databases node. It will look something like
<key name="Newdb">
<value name="Accessed">A32BFE2475B3E440</value>
<value name="CaseSensitiveRole">0</value>
<value name="CharacterSet"/>
<value name="Created">7DB1327474B3E440</value>
<value name="DatabaseFiles">D:\aaad7\Interbase\NEWDB.GDB</value>
<value name="Encrypted">0</value>
<value name="EUA Enabled">0</value>
<value name="Password">096e6376786a78726d82</value>
<value name="Role"/>
<value name="Save DBAlias">1</value>
<value name="SEP"/>
<value name="Use DBAlias">0</value>
<value name="Username">SYSDBA</value>
<value name="UseSSL">0</value>
</key>
Previous answer
Try this:
In IBConsole, click the Local Server node and select its Databases node
Right-click the Databases node and select Add
In the Add database and Connect pop-up, click the button to the RHS of the File: field
From there, you should be able to navigate to and select your new database.
Tested with Interbase XE7 IBConsole.
If that doesn't work for you, exactly which step fails and how?
It seems I find the answer. Code creates database file and add database on InterBase server. It can be checked by isql utility. Strange that IBConsole and IBExpert don't show all databases on server in current moment but only created by the same tool as I understand.

Mondrian saiku - vertica query translation error

Hi trying to use saiku with vertica.
Vertica has the concept of db -> schemas -> tables. So in the xml file, instead of the table name, I am giving schemaName.tableName
<?xml version="1.0"?>
<Schema name="Sales" metamodelVersion='3.6' quoteSql='false'>
<Cube name="Sales" defaultMeasure="sales">
<Table name="schemaName.factName"></Table>
<Dimension name="date_mysql">
<Hierarchy hasAll="true">
<Level name="date" column="date" type="Date" uniqueMembers="false"/>
</Hierarchy>
</Dimension>
<Measure name="sales" aggregator="sum" column="sales" formatString="#,###" />
<Measure name="orders" aggregator="sum" column="orders" formatString="#,###" />
</Cube>
</Schema>
This seem to work, and mondrian is able to pick up the measure and dimension properly. The problem is the SQL query generated is syntactically wrong
select "schemaName"."tableName"."date" as "c0"
from "schemaName"."tableName" as "schemaName"."tableName"
group by "schemaName"."tableName"."date"
order by CASE WHEN "schemaName"."tableName"."date" IS NULL THEN 1 ELSE 0 END, "schemaName"."tableName"."date" ASC
There are two problems here.
Vertica treats double quotes as any other character and hence "tableName" and tableName are distinct. ( quoteSql='false' doesnt work as Iam using metamodel 3.6)
Mondrian seems to generate alias from the the table name specified (which here is schema.table) which goes wrong here.
Is there any other way to mention schema? And how to get rid of the double quotes?
The table tag carries a schema attribute as well.(Thanks to Paul Stoellberger for pointing out) So
<Table name="factName" schema="schemaName"></Table>
This takes care of the dialect and quoting problems
http://mondrian.pentaho.com/documentation/xml_schema.php#Table

Informix select first 250000 then the last 250000 records in a table

I work on a CFML script to backup some data in a CSV file from Informix database. The problem is the table has many records 286906 and my scripts timeouts (even I set it not to), the best I could successfully was 260000 with:
SELECT FIRST 260000
APE1, APE2, CALLE, CODPOSTAL, DNI, FCADU, FENACI, LOCALIDAD, NOMBRE, NSS, PROV, TELEFONO
FROM
mytable WHERE FCADU IS NOT NULL AND FENACI IS NOT NULL
is there any way to select the rest of 260000 and then the rest?
I tried with:
SELECT SKIP 260000 FIRST 520000
APE1, APE2, CALLE, CODPOSTAL, DNI, FCADU, FENACI, LOCALIDAD, NOMBRE, NSS, PROV, TELEFONO
FROM
mytable WHERE FCADU IS NOT NULL AND FENACI IS NOT NULL
but I get Error Executing Database Query. A syntax error has occurred.
You can use the Unload statement for creating a file from database:
UNLOAD TO 'mytable.txt' SELECT * FROM mytable;
Maybe that this not works in CFML environment. So you can create a stored procedure which unloads your data.
See unload statement in stored procedure
Is it your script timing out or your database connection? From your question it sou ds to me like its not the coldfusion template that is timing out but the cfquery connection to the database. There is a timeout attribute for the cfquery tag. However apparently it is not reliable a better option is to configure the timout in the advanced section of the datasource within the coldfusion administrator.
Charlie Arehart blogged about this feature here:
http://www.carehart.org/blog/client/index.cfm/2010/7/14/hidden_gem_in_cf9_admin_querytimeout

Resources