I am working for a project using the BI tools of Pentaho: Pentaho Mondrian.
I am experiencing problems with the use of aggregate tables: for make aggregate tables I used Pentaho Aggregate Designer and I successfully created the DDL, populated the new tables and published the schema into Pentaho.
I am using Saiku but I'd see, for the response times, that it doesn't use the aggregate tables even if it's specified in the schema.
Of course I changed the values in every mondrian.properties file,
mondrian.rolap.aggregates.Use=true
mondrian.rolap.aggregates.Read=true
but no way for make it working!
I attached you a screenshot of the aggregate designer and another of the result using Pentaho and Saiku (you can see that the query was executed in more than 10s when the aggregate table has only 9 tuples)
Do you have some ideas or did you experienced this problem and you solved?
Thank you in advance!
It took me also a time to make aggregate tables work. Here is one quick tip if you face this issue:
Use standalone version of Saiku server.
Deploy your OLAP schema on Saiku.
Enable Special Log File specifically for Mondrian SQL Statements in saiku/tomcat/webapps/saiku/WEB-INF/clasess/log4j.xml
Run Saiku, execute same queries where you expect the aggregate tables will be used.
Check the log saiku/tomcat/logs/mondrian.sql to see if your aggregate tables are really used or not.
This helped me a lot when finding the solution for my problem in OLAP schema (wrong db column used for aggregate dimension).
Related
We started using Snowflake data warehouse in our project recently and encountered an issue. The table and view creation scripts were uploaded to Git repo and executed through Jenkins. For any updation (columns addition or deletion) of the tables creation scripts in snowflake, We had to drop and recreate the tables in snowflake. This is causing data loss from tables which is undesirable.
Every time we drop and recreate the tables, data is getting lost and we had to manually load all the data to newly created tables. This is both time consuming and causing unwanted down time.
Our table creation scripts are generic for all environments and were previously designed for HIVE. The same were extended to snowflake and we are facing issues only during any schema changes to the main snowflake tables.
The ALTER command is also not solving our problem when we add new columns to tables. I am looking for any tool or process which can deploy the changes to tables or can modify the tables without deleting the underlying data.
I considered Oracle SQL developer data modeler (SDDM) after going through Snowflake documentation but I couldn't find any proper method to change the already created tables. All I found was how to connect to snowflake form the tool and do an import from there. Couple other tools like Sqitch and Dataedo were mentioned in the chain of snowflake documentation but they donot seem to meet my requirement.
The link to snowflake documentation community is as below.
https://snowflakecommunity.force.com/s/question/0D50Z00007v6qQbSAI/what-schema-management-tools-are-there-for-snowflake
Could anyone please suggest any tools or process to solve my issue. thanks in advance
Assuming that you want to automate the creation of Snowflake Alter Table statements, you might want to look at FlywayDB. It supports capturing DDL changes for Snowflake with both a free and paid version. It was made by Redgate which has a long history of SQL Server schema comparisons and migrations.
I need to add tag reporting capability to a collection of custom SSRS reports which query TFS_Warehouse (and in one case I had to query the operational store to gather test case steps). These reports all use a SQL Server datasource connected to my custom TFS_Warehouse_Extensions database.
If this sounds familiar, I asked this question yesterday and got a wonderful response... then I discovered we upgraded from 2013 to 2015 last week and dbo.workitemsare is gone.
I am using VS 2015 and am more of a database developer than a C# programmer (I know just enough to be dangerous). Is there any way I can get tags from TFS 2015 workitems into my SSRS reports?
Edit: the proposed duplicate answer is not exactly the same problem. Whether or not some work items views went missing is ancillary. My requirement is for a way to query TFS tags in SSRS. So far I consider this unanswered in either thread since no one has proposed a solution that meets the requirement.
#Cyndi, I'm a Program Manager with the reporting team. Currently reporting with Tags is not supported aside from queries with the query editor. We do have a new reporting solution we're working on and reporting on tags will be supported. I don't have an exact date for the release yet, but see this blog post for some details. We'll have more announcements to make this summer.
One totally diffrent way would be to use Excel for reporting.
You build a query in TFS and then connect Excel to TFS and use the query. The functionality is kind of limited, but you can use Excels features to make great reports.
One problem with Excel is that you can't (you shouldn't) add further fields to the table you got from TFS. When you update the data, Excel will lock up and nothing happens.
If you need to use custom fields for your reporting, you should create a second table, where all the data is copied to. This can be as well automated with VBA.
That data can be analysed directly in the table or with a Pivot Table or visualised a Pivot Chart.
With the use of the Pivot feature Excel is able to create powerful reports.
It's not a good idea to query the operational store. This may cause some problems for normal usage.
You should NEVER write reports directly against the WorkItem* tables
in the collection database, since this is an operational store and is
100% unsupported and can cause performance problems for normal usage.
Source: How to query Work Items using SQL on the Relational Warehouse
I'm working against a DB2 database with a DotNet application. Some parts of the system are build and maintained in AS400 LANSA, other parts are DotNet.
We curently have the issue that we are maintaining a lot of summary table that need to be updated with values of many different tables. This causes our data to be out of sync with each other all the time and we need to run script each night checking and correcting errors.
These tables are required since they claim it is impossible to make join or use views in LANSA
Is this correct? Are they alternatives i can supply for them to avoid these problems?
If you use LANSA with RDMLX, then you can use SELECT_SQL to incorporate JOIN's as needed.
By Views, do you mean logical files? Those get used as needed by the database engine automatically. If you mean a VIEW created off multiple physical files, then these can be replicated in LANSA using pre-determined Join Fields. Have a look at the LANSA documentation about these.
You could use something like this:
#MYSQLST := ('SELECT {write your SQL select statement here}') Select_Sql Fields({my fields}) Using(#MYSQLST)
Worth reviewing the LANSA documentation on the use of Free Format SELECT_SQL
I have tables which have millions of records. I need to load these records as nodes in neo4j.
Please help me out on how to do it as I'm new to neo4j.
It is quite easy, just map your entities that should become nodes into a set of csv files and the connections that should become relationships in another set of files.
Then run them with my batch-importer: https://github.com/jexp/batch-import/tree/20#binary-download
import.sh nodes1.csv,nodes2.csv rels1.csv,rels2.csv
Add types and index information to the headers and the batch.properties config file as needed.
You can use the batch-importer for the initial inserter but also subsequent updates (but the database has to be shut-down for that).
It is pretty easy to connect to your existing database using its driver and then extract the information of the right shape and kind and insert it into your graph model,
Either using Cypher statements with parameters or the embedded, transactional Java API for ongoing updates.
See: http://jexp.de/blog/2013/05/on-importing-data-in-neo4j-blog-series/
You can export to CSV and import it into node (probably wont work well since you have millions of records)
You can write a program to do it (this is what I am currently working on).
This also depends on what programming languages you know... but the bottom line is, because no two databases are created equally (unless on purpose), it's very difficult to create a catch-all solution for migrating data from SQL to Neo.
The best way that I've discovered so far is to create a program that queries the tables in the database, finds all related tables (i.e. foreign keys), and imports all those table rows into Neo, labeling the nodes using the Table name, then process the foreign keys as relationships.
It's not easy. I've been working on something for my database here for a week or so now... but I'm close!
Hi
I have the database with huge data. there are several master-detail relationships within it. so i just wanted to build the Views / stored procedure so that data fetching get much faster. I know I can use Index. but i saw it is a limitation in sql server express version. where as i am using express version. so how to perform this in express version. please guide me .
Creating a stored procedure won't speed up query execution.
Making sure that your foreign keys are indexed is the first step.
The definite answer to the question of query performance will give you a look at query execution plan. Make sure you don't have any Table Scans / Table Seeks.