I am trying to write a SSIS package to transfer data from a flat file into my database. My lookup column in the DB table has the names "BUY", "SELL", "BUY COVER", "SHORT SELL", and "POSITION ADJUSTMENT". The data I am mapping to this table has "BUY", "SELL", "COVER", and "SHORT", respectively.
How do I get the mappings to recognize that "BUY COVER"/"COVER" and "SHORT SELL"/"SHORT" should be matched up? How do I transform the incoming data to be applicable to my DB requirements?
Using Visual Studio Business Intelligence Development Studio 2008 and SQL Server 2008.
You must provide those mappings, much as I described in this answer. For all of your possible inputs, you need to create a translation to the reference value---same as if you were doing this in SQL or anything else.
I ended up doing something different to solve this problem. Since I just wanted to be able to map from columns in the flat file to the database, I did the following and it works just fine.
In the Lookup Transformation Editor I inserted the following query into the SQL Query option for the lookup:
select cast (TradeTypeId as int) as TradeTypeId,
TradeType =
case
when TradeTypeId = 3 then 'COVER'
when TradeTypeId = 4 then 'SHORT'
else TradeTypeName
end
from dbo.sct_TradeType
where TradeTypeId<> 9
The "TradeType" holds the values "BUY", "SELL", "BUY COVER", "SHORT SELL", and "POSITION ADJUSTMENT". I am not using "POSITION ADJUSTMENT" (id =9) so it's eliminated from the query at the end. This converted my file column names to match my database names and everything works!
Related
I can query using Cypher in Neo4j from the Panama database the countries of three types of identity holders (I define that term) namely Entities (companies), officers (shareholders) and Intermediaries (middle companies) as three attributes/columns. Each column has single or double entries separated by colon (eg: British Virgin Islands;Russia). We want to concatenate the countries in these columns into a unique set of countries and hence obtain the count of the number of countries as new attribute.
For this, I tried the following code from my understanding of Cypher:
MATCH (BEZ2:Officer)-[:SHAREHOLDER_OF]->(BEZ1:Entity),(BEZ3:Intermediary)-[:INTERMEDIARY_OF]->(BEZ1:Entity)
WHERE BEZ1.address CONTAINS "Belize" AND
NOT ((BEZ1.countries="Belize" AND BEZ2.countries="Belize" AND BEZ3.countries="Belize") OR
(BEZ1.status IN ["Inactivated", "Dissolved shelf company", "Dissolved", "Discontinued", "Struck / Defunct / Deregistered", "Dead"]))
SET BEZ4.countries= (BEZ1.countries+","+BEZ2.countries+","+BEZ3.countries)
RETURN BEZ3.countries AS IntermediaryCountries, BEZ3.name AS
Intermediaryname, BEZ2.countries AS OfficerCountries , BEZ2.name AS
Officername, BEZ1.countries as EntityCountries, BEZ1.name AS Companyname,
BEZ1.address AS CompanyAddress,DISTINCT count(BEZ4.countries) AS NoofConnections
The relevant part is the SET statement in the 7th line and the DISTINCT count in the last line. The code shows error which makes no sense to me: Invalid input 'u': expected 'n/N'. I guess it means to use COLLECT probably but we tried that as well and it shows the error vice-versa'd between 'u' and 'n'. Please help us obtain the output that we want, it makes our job hell lot easy. Thanks in advance!
EDIT: Considering I didn't define variable as suggested by #Cybersam, I tried the command CREATE as following but it shows the error "Invalid input 'R':" for the command RETURN. This is unfathomable for me. Help really needed, thank you.
CODE 2:
MATCH (BEZ2:Officer)-[:SHAREHOLDER_OF]->(BEZ1:Entity),(BEZ3:Intermediary)-
[:INTERMEDIARY_OF]->(BEZ1:Entity)
WHERE BEZ1.address CONTAINS "Belize" AND
NOT ((BEZ1.countries="Belize" AND BEZ2.countries="Belize" AND
BEZ3.countries="Belize") OR
(BEZ1.status IN ["Inactivated", "Dissolved shelf company", "Dissolved",
"Discontinued", "Struck / Defunct / Deregistered", "Dead"]))
CREATE (p:Connections{countries:
split((BEZ1.countries+";"+BEZ2.countries+";"+BEZ3.countries),";")
RETURN BEZ3.countries AS IntermediaryCountries, BEZ3.name AS
Intermediaryname, BEZ2.countries AS OfficerCountries , BEZ2.name AS
Officername, BEZ1.countries as EntityCountries, BEZ1.name AS Companyname,
BEZ1.address AS CompanyAddress, AS TOTAL, collect (DISTINCT
COUNT(p.countries)) AS NumberofConnections
Lines 8 and 9 are the ones new and to be in examination.
First Query
You never defined the identifier BEZ4, so you cannot set a property on it.
Second Query (which should have been posted in a separate question):
You have several typos and a syntax error.
This query should not get an error (but you will have to determine if it does what you want):
MATCH (BEZ2:Officer)-[:SHAREHOLDER_OF]->(BEZ1:Entity),(BEZ3:Intermediary)- [:INTERMEDIARY_OF]->(BEZ1:Entity)
WHERE BEZ1.address CONTAINS "Belize" AND NOT ((BEZ1.countries="Belize" AND BEZ2.countries="Belize" AND BEZ3.countries="Belize") OR (BEZ1.status IN ["Inactivated", "Dissolved shelf company", "Dissolved", "Discontinued", "Struck / Defunct / Deregistered", "Dead"]))
CREATE (p:Connections {countries: split((BEZ1.countries+";"+BEZ2.countries+";"+BEZ3.countries), ";")})
RETURN BEZ3.countries AS IntermediaryCountries,
BEZ3.name AS Intermediaryname,
BEZ2.countries AS OfficerCountries ,
BEZ2.name AS Officername,
BEZ1.countries as EntityCountries,
BEZ1.name AS Companyname,
BEZ1.address AS CompanyAddress,
SIZE(p.countries) AS NumberofConnections;
Problems with the original:
The CREATE clause was missing a closing } and also a closing ).
The RETURN clause had a dangling AS TOTAL term.
collect (DISTINCT COUNT(p.countries)) was attempting to perform nested aggregation, which is not supported. In any case, even if it had worked, it probably would not have returned what you wanted. I suspect that you actually wanted the size of the p.countries collection, so that is what I used in my query.
Symptoms:
Searching for a single word (i.e. "Snap") works
Searching for another word contained in the same field (i.e. "On") also works
Searching for "Snap On" at the same time returns 0 results, even though it shouldn't.
The setup:
SQL Server 2008 R2 with Advanced Features
nopCommerce 3.0
Things I have done:
I added the Product.MetaKeywords column to the full text search catalog
I added a bit into the Stored Procedure that performs the search to search through the MetaKeywords
Now the nopCommerce boards are fairly slow, but I'm positive the problem is within the SQL Stored Procedure anyway, so I figured I would ask for some SQL Server help here, even if you aren't familiar with the nopCommerce web app, you may have some information you can help me with.
The stored procedure in question is too large to post entirely here, but basically it dynamically adds "OR" or "AND" in between the keyword searches to generate the phrase used in a Contains clause. It selects through several unions various searchable fields by using Contains.
Here is the bit I added into the stored procedure
SET #sql = #sql + '
UNION
SELECT p.Id
FROM Product p with (NOLOCK)
WHERE '
IF #UseFullTextSearch = 1
SET #sql = #sql + 'CONTAINS(p.[MetaKeywords], #Keywords) '
ELSE
SET #sql = #sql + 'PATINDEX(#Keywords, p.[MetaKeywords]) > 0 '
#Keywords, at this point, if I am reading the procedure correctly, has a value of: "Snap* AND On*"
I don't understand why my query of "Snap On" returns 0 results, but "Snap" and "On" individually work fine.
The minimum search length is set to 1, so it's not that.
I should add that searching for "Snap* OR On*" works, but I cannot use OR because then searching for "Snap On" will also return "Snap Dragon" and other unrelated things.
--EDIT--
The problem wasn't any of that. I got some advice elsewhere and the problem was actually the stoplist. I managed to fix my issue simply by changing the stoplist on the product table from <system> to <off>.
To do this, follow these steps.
browse to your table in SQL Server management studio
Right click on the table and select "Full-Text Index"
Select "Properties" under "Full-Text Index"
In the "General" Tab, change "Full-Text Index Stoplist" to <off>
I had to do it this way because I was unable to get the transact SQL to work. It kept telling me there was no such object as the table I was attempting to modify. If anyone can provide any insight on how the Alter fulltext index statement works, I'm interested, because I was following the example on the MSDN page to the T and it just kept telling me there was no such object named Product.
The asterisk is not a plain old wildcard. If you are using it anywhere other than at the end of a search term, you're probably not using it correctly. See answers to a similar question
SQL Contains Question
In your case, each search term must be quoted separately. See this example from the docs http://technet.microsoft.com/en-us/library/ms187787(v=sql.90).aspx
SELECT Name
FROM Production.Product
WHERE CONTAINS(Name, '"chain*" OR "full*"');
I am trying to do something like this:
merge MembershipTEST as T
using (select OrganisationID, Name From MembershipPending) as S
on T.OrganisationID = S.OrganisationID
and T.Name = S.Name
when not matched then
insert (MembershipID,OrganisationID, Name)
values(
(EXEC [dbo].[spGetNextIntKeyByTableName]
#PKColName = 'MembershipID',#TableName = 'FWBMembership'),
S.OrganisationID,
S.Name );
Bascially the identitykey is from a sp
Is it possible?
Update 1: Answer is NO
read the online doc http://msdn.microsoft.com/en-us/library/bb510625%28v=sql.105%29.aspx
VALUES ( values_list) Is a comma-separated list of constants,
variables, or expressions that return values to insert into the target
table. Expressions cannot contain an EXECUTE statement.
Build an SSIS package. Just create a new Data Flow task. Use two OLEDB sources make one of them execute the stored procedure and the other one select from the table you are looking to MERGE with. Make sure they are both ordered by the same thing. Go into advanced settings for each OLEDB source and set is sorted to true and then set the sort values for the items you are ORDERing BY. Then move both data flows to to a MERGE JOIN. Then send the Data flow to a Conditional Split and set as INSULL(OrganisationID). Use the resulting data flow to go to an OLEDB destination.
Sorry for the lack of visuals I will add them later when I'm on lunch hour to busy to add them now.
I have merged two Fusion Tables together on the key "PID". Now I would like to do a SELECT query WHERE PID = "value'. The error comes back that no column with the name PID exists in the table. A query for another column gives this result:
"kind": "fusiontables#sqlresponse",
"columns": [
"\ufeffPID",
"Address",
"City",
"Zoning"
],
"rows": [
[
"001-374-079",
"# LOT 15 MYSTERY BEACH RD",
"No_City_Value",
"R-1"
],
It appears that the column name has been changed from "PID" to "\ufeffPID", which no matter how many attempts to get the syntax to read a GET Url, I keep getting an error.
Is there any limitation with querying on the key of a merged table? Since I cannot seem to get the name correct for the column a work around would be to use the Column ID but that does not seem to be an option either. Here is the URL:
https://www.googleapis.com/fusiontables/v1/query?sql=SELECT 'PID','Address','City','Zoning' FROM 1JanYNl3T45kFFxqAmGS0BRgkopj4AS207qnLVQI WHERE '\ufeffPID' = 001-493-078&key=myKey
Cheers
I have no explanation for \ufeff in there; that's the Unicode character 'ZERO WIDTH NO-BREAK SPACE', so it's conceivable that it's actually there in the column name because it would be invisible in the UI. So, first off I would recommend changing the name in the base tables and see if that works.
Column IDs for merge tables have a different form than for base tables. An easy way to get them is to add the filters of interest to one of your tabs (any type will do) and then do Tools > Publish. The top text ("Send a link in email or IM") has a query URL that has what you need. Run it through a URL decoder such as http://meyerweb.com/eric/tools/dencoder/ and you'll see the column ID for PID is col0>>0.
Rod
INFORMIX-SQL 7.3 Perform Screens:
According to documentation, in an "after editadd editupdate of table" control block, its instructions are executed before the row is added or updated to the table, whereas in an "after add update of table" control block, its instructions are executed after the row has been added or updated to the table. Supposedly, this would mean that any instructions which would alter values of field-tags linked to table.columns would not be committed to the table, but field-tags linked to displayonly fields will change?
However, when using "after add update of table", I placed instructions which alter values for field-tags linked to table.columns and their displayed and committed values also changed! I would have thought that an "after add update of table" would only alter displayonly fields.
TABLES
customer
transaction
branch
interest
dates
ATTRIBUTES
[...]
q = transaction.trx_type, INCLUDE=("E","C","V","P","T"), ...;
tb = transaction.trx_int_table,
LOOKUP f1 = ta_days1_f,
t1 = ta_days1_t,
i1 = ta_int1,
[...]
JOINING *interest.int_table, ...;
[...]
INSTRUCTIONS
customer MASTER OF transaction
transaction MASTER OF customer
delimiters ". ";
AFTER QUERY DISPLAY ADD UPDATE OF transaction
if z = "E" then let q = "E"
if z = "C" then let q = "C"
if z = "1" then let q = "E"
[...]
END
Is 'z' a column in the transaction table?
Is the trouble that the value in 'z' is causing a change in the value of 'q' (aka transaction.trx_type), and the modified value is being stored in the database?
Is the value in 'z' part of the transaction table?
Have you verified that the value in the DB is indeed changed - using the Query Language option or a simple (default) form?
It might look as if it is because the instruction is also used AFTER DISPLAY, so when the values are retrieved from the DB, the value displayed in 'q' would be the mapped values corresponding to the value stored in 'z'. You would have to inspect the raw data to hide that mapping.
If this is not the problem, please:
Amend the question to show where 'z' comes from.
Also describe exactly what you do and see.
Confirm that the data in the database, as opposed to on the screen, is amended.
Please can you see whether this table plus form behaves the same for you as it does for me?
Table Transaction
CREATE TABLE TRANSACTION
(
trx_id SERIAL NOT NULL,
trx_type CHAR(1) NOT NULL,
trx_last_type CHAR(1) NOT NULL,
trx_int_table INTEGER NOT NULL
);
Form
DATABASE stores
SCREEN SIZE 24 BY 80
{
trx_id [f000]
trx_type [q]
trx_last_type [z]
trx_int_table [f001 ]
}
END
TABLES
transaction
ATTRIBUTES
f000 = transaction.trx_id;
q = transaction.trx_type, UPSHIFT, AUTONEXT,
INCLUDE=("E","C","V","P","T");
z = transaction.trx_last_type, UPSHIFT, AUTONEXT,
INCLUDE=("E","C","V","P","T","1");
f001 = transaction.trx_int_table;
INSTRUCTIONS
AFTER ADD UPDATE DISPLAY QUERY OF transaction
IF z = "E" THEN LET q = "E"
IF z = "C" THEN LET q = "C"
IF z = "1" THEN LET q = "E"
END
Experiments
[The parenthesized number is automatically generated by IDS/Perform.]
Add a row with data (1), V, E, 23.
Observe that the display is: 1, E, E, 23.
Exit the form.
Observe that the data in the table is: 1, V, E, 23.
Reenter the form and query the data.
Update the data to: (1), T, T, 37.
Observe that the display is: 1, T, T, 37.
Exit the form.
Observe that the data in the table is: 1, T, T, 37.
Reenter the form and query the data.
Update the data to: (1), P, 1, 49
Observe that the display is: 1, E, 1, 49.
Exit the form.
Observe that the data in the table is: 1, P, 1, 49.
Reenter the form and query the data.
Observe that the display is: 1, E, 1, 49.
Choose 'Update', and observe that the display changes to: 1, P, 1, 49.
I did the 'Observe that the data in the table is' steps using:
sqlcmd -d stores -e 'select * from transaction'
This generated lines like these (reflecting different runs):
1|V|E|23
1|P|1|49
That is my SQLCMD program, not Microsoft's upstart of the same name. You can do more or less the same thing with DB-Access, except it is noisier (13 extraneous lines of output) and you would be best off writing the SELECT statement in a file and providing that as an argument:
$ echo "select * from transaction" > check.sql
$ dbaccess stores check
Database selected.
trx_id trx_type trx_last_type trx_int_table
1 P 1 49
1 row(s) retrieved.
Database closed.
$
Conclusions
This is what I observed on Solaris 10 (SPARC) using ISQL 7.50.FC1; it matches what the manual describes, and is also what I suggested in the original part of the answer might be the trouble - what you see on the form is not what is in the database (because of the INSTRUCTIONS section).
Do you see something different? If so, then there could be a bug in ISQL that has been fixed since. Technically, ISQL 7.30 is out of support, I believe. Can you upgrade to a more recent version than that? (I'm not sure whether 7.32 is still supported, but you should really upgrade to 7.50; the current release is 7.50.FC4.)
Transcribing commentary before deleting it:
Up to a point, it is good that you replicate my results. The bad news is that in the bigger form we have different behaviour. I hope that ISQL validates all limits - things like number of columns etc. However, there is a chance that they are not properly validated, given the bug, or maybe there is a separate problem that only shows with the larger form. So, you need to ensure you have a supported version of the product and that the problem reproduces in it. Ideally, you will have a smaller version of the table (or, at least, of the form) that shows the problem, and maybe a still smaller (but not quite as small as my example) version that shows the absence of the problem.
With the test case (table schema and Perform screen that shows the problem) in hand, you can then go to IBM Tech Support with "Look - this works correctly when the form is small; and look, it works incorrectly when the form is large". The bug should then be trackable. You will need to include instructions on how to reproduce the bug similar to those I gave you. And there is no problem with running two forms - one simple and one more complex and displaying the bug - in parallel to show how the data is stored vs displayed. You could describe the steps in terms of 'Form A' and 'Form B', with Form A being Absolutely OK and Form B being Believed to be Buggy. So, add a record with certain values in Form B; show what is displayed in Form B after; show what is stored in the database in Form A after too; show that they are not different when they should be.
Please bear in mind that those who will be fixing the issue have less experience with the product than either you or me - so keep it as simple as possible. Remove as many attributes as you can; leave comments to identify data types etc.