Which is better to use for rendering large amount of data with better performance, ag-grid or angular material table?
I'm trying to render 70,000 to 1,50,000 of data. I will also be performing certain operations like sorting filtering.
I have used both of them. So, I can confidently say that Ag-grid is much better than material table as it is very easy to use and efficient as well, but make sure that whatever features you are planning to use are available with community edition. Otherwise you need a license for enterprise edition features.
For more details of ag-grid, you can checkout this url: https://www.ag-grid.com/angular-data-grid/licensing/
Related
I have a Rails application which is mainly used to show data to the users through dashboards.
The application is currently running on a PostgreSQL database.
All the history values stored so far are in the form [date,variable,value].
The records can be stored every 5 minutes, and the number of variables I'm tracking is high, aka, millions of records in the history table.
Now I need to do operations in between variables. So let's say I have a variable A and a variable B, I would need:
the value of A/B.
the history of A/B calculated every t (which can be 5 minutes/1 day/1 week).
Currently I am doing everything within the Rails environment, but I know that Ruby isn't a fast language so I'm planning a change.
In the near future I would also need to calculate derivatives and integrals so I was thinking that I might need a dedicated/decoupled services that only does data crunching.
What would be a good tool/language that allows me to do complex math operations on large set of data?
What would be a good tool/language that allows me to do complex math operations on large set of data?
Python and numpy/scipy are what you need! I'm not trying to sell some python to a ruby afficionado, but I can vouch for these packages, they're awesome.
It's close to matlab where you manipulate native vector/matrix data types, using functions that support vectorization. It can use openmp for parallelism, there is also numexpr. It is both very fast and very easy to use, plus the library is enormous, here is an example.
Thanks to peufeu answer, I could find these guys: http://sciruby.com/
I am building a business intelligence web app in php 5 that display informations retrieved from a datawarehouse highly normalized (60+ tables in mysql).
We use MODx as our CMF to organize the code. So far the code is mainly procedura, each page is essentially composed of a bunch of sql query directly in the php code (Snippet as in the MODx terminology) and code to display the info in tables and graphically.
We are in the process of creating objects for our main components and put the sql queries there and use PDO. It is easy to do when the query map to a real object of the domain.
For more BI (aggregation with subqueries, join on 5+ tables) or search oriented query, I find it more difficult to see how to replace the dynamically created sql. For example, we have a search functionality in the web app with a lot of criteria. Depending on which criteria are selected, the php code add or remove tables to join, subqueries and change the 'where' clause.
Do you think an ORM or Stored Proc could improve performance/quality of code in that context ?
Is our model (60+ tables highly normalized) too complicated to be directly accessed from the web app and a kind of datamart (basically denormalized view of the data) would bring more benefits than ORM ?
This question is related to : stored-procedures-or-or-mappers
The bottleneck would surely be the level of normalization - if the option is available to you, adopting a more star-schema style DWH would greatly increase performance as it pre-prepares the data for consumption by your BI app.
I have a few apps written in ruby on rails and like any good developer I want high quality data about my site, such as measuring the number of new user accounts per day. I'm in the process of writing my own analytics tools, but I feel like i'm re-inventing the wheel. Are there any plugins or gems that could help me pull this data and display it quickly (graphs are a plus)?
If not, what types of features would you want in such a tool (i'll put a plugin on github if my code is good enough)?
Update:
To clarify a bit, i'm looking for business level-analytics. I already use google-analytics for my site traffic, and active-scaffold to get an admin page, right now my application has users which generate tickets and can create surveys, i'm interested in general trends in my application and by graphing new & existing user numbers versus new tickets and new surveys i can get the info that I want. I like to get general numbers, so i'm pulling all the users for the last 30 days, and then iterating over them to count how many i get per day...then i'm saving that to an array and plotting versus tickets, etc. Right i'm doing this using a home brew library which isn't very efficient, and before I put time/energy into making it better I want to make sure i'm not duplicating an existing set of tools. Or writing un-needed code.
If you post how you personally do this, and the answer is at least intelligible i'll be happy to give you a karma bump for your time.
You have three options that are all fairly easy to implement:
Google Analytics
Just include a small javascript snippet in the footer of your page and you get meaningful data about your hits/traffic. This is extremely easy, and will provide traffic information, but nothing about the internal workings about your applications.
New Relic: RPM
New Relic RPM is a service that comes in the form of a plugin. There is a free version, which gives you a (useful) taste of the features it can provide. This plugin will give you hardcore rails analytics. It will tell you what percentage of a request to a controller is spent in the model, in the view, etc. It will tell you how long each SQL call takes. This is great for optimizing your application.
ActiveScaffold
While not in and of itself an administrative tool, ActiveScaffold fits the bill quite nicely. Just create an admin namespace and create ActiveScaffolds for all your models/resources. This lets you see the data in an easy to use way, get simple counts of your rows (to see how many users you have, for example). This is a very easy setup, with little overhead.
Edit to reply to the OP Edit
There are no gems/plugins that I'm aware of that provide business-level analytics that you seem to want, as they are specialized associations between models that can't be predicted. The best bet, in my opinion, would be to roll your own solution that provides the data you want.
Probably the easiest way is to stick with good ol' Google Analytics. I'm pretty sure there are tools for more specific needs, but for general purpose analytics they are probably the best.
I am investigating moving a thick client SQL based Delphi application to Multi Tier thin clients, and have been looking at using Datasnap in Delphi 2010. I have worked through the White Paper written by Bob Swart and extended this further.
My main question really is that I want to make the server side efficient in terms of connections and SQL Queries due to multiple queries being run and remaining open at the same time to interrogate data, can anyone point me in a direction for guidance on how to design a real world Datasnap Server application, as the demo's don't go into enough detail.
Thanks
Matt
You have to decide in your design:
(Mid-Server) Will manage session or client will identify their session each connection (stateful vs stateless)
(Mid-Server) How much cached data you desire to have. You can cache just some annoying very stable tables and only querying them when they change (if all modifications is through mid-server, if not, you need something like an arbitrary mark - a GUID, a counter - on the table to match if data changed).
(Client/Mid-Server) If your client will always get a full collection of data or just fragments of the collection.
(ex.: a product can have an categoryId column, which is a FK to a Category table. You can send both all the time or the client can request only the product data).
(Mid-Server/RDBMS)You maybe have to provide some form of custom search. If you have an clue of the most used search conditions, you can provide (if needed) the index covering to do that.
(Mid-Server/RDBMS)Don't bring great datasets to mid-server, unless you plan on do some aggresive caching of data and/or have some good use for them. Mid-Server is just another client application to the RDBMS - if both are on the same machine, you can enter in a memory/CPU/IO competition with the RDBMS.
(Mid-Server/RDBMS)Execute your business rules on the Mid-Server, is the mantra of many purists out there. For me, equilibrium is key. Let's say the RDBMS have 2000 Stored Procedures (I'm not exagerating, there's real business with such a number of SPs) and your Mid-Server can make an excellent work on 1500 of that 2000 problems solved by SPs, GREAT and do that. But if the last 500 can be a change for worst, let them alone. It can be a bigger hassle than the 1500.. So mix the two, and make those 500 an software project to an other version.
(Mid-Server/RDBMS) What I said for Stored Procedures can also be applied to triggers and other any other kind of RDBMS server features that can make your live easier.
(Mid-Server/DataSnap)Most of the crude details can be let to dataset provider. But learn to leverage the OnBeforeUpdateRecord event to do custom processing when needed.
(Mid-Server/DataSnap)You know you can create a ClientDataset only with modified data by
this kind of code below? You cannot imagine how useful it can be... :-)
Cds_New.Data := Cds_Updated.Delta;
I'm looking for recommendations for a spreadsheet-like input facility to sit in an ASP MVC environment.
The client currently has a large number of very complex interlinked shared spreadsheets (which they are effectively running 90% of their core business from) for collecting and processing information. They wish to move this to a web application and require ASP MVC. They realise that they will not be able to display as much information on screen as they do currently with their spreadsheets, so a 40 x 60 grid should suffice in most cases. Of this they'll be a limited number of cells for data entry that will immediately update other cells in the grid using various spreadsheet-like formula. The grid must be AJAX enabled.
The quality of the user-interface is of primary concern here. As inevitably there will be a certain amount of resistance to move to database/web (and this project is a pilot anyway) the system must be as slick as possible. Almost as important is ease of implementation - the final system will be quite large so the quicker it is possible to configure the grid the better.
Either Open Source or commercial would be fine. HTML/Javascript, Silverlight and Flex implementations can all be considered.
I initially asked a similar question a year ago (it's taken that long for the client to agree the project) but I'm sure options have changed since then and our environment is now better defined.
I think Grapecity Spread will fit the bill , it can be easily built with MVC pattern and now it also supports the Razor View Engine .There is already a blog which details how to use Grapecity Spread with MVC , you can go through this here
http://www.gcpowertools.info/2011/12/using-grapecity-spread-for-net-with-mvc.html
Microsoft Silverlight. It is almost certainly your best bet for a rich line of business application with web deployment. It will allow you to utilise a consistent code base across your back-end and front-end component.
Whilst a number of commercial datagrid packages exist (Telerik, etc), I'd suggest using the default Datagrid component that is available, and understanding fully the databinding and templating options available.
Check out YUI's datatable, maybe what you need
http://developer.yahoo.com/yui/examples/datatable/dt_cellediting.html
I have used it and its great, very developer friendly and supports pagination.
When asked for a multi-line editable grid, I've done it two ways.
Dropped a SL control onto the page. This was incredibly easy.
A lot of javascript. Double-clicking a row made a row editable with several textboxes to span the gap. I don't think is what you're looking for, though.
For something quick and easy have a look at JqGrid demos to see if it can do what you want
http://www.trirand.com/blog/jqgrid/jqgrid.html
JQuery is already in MVC and being javascript it will work without plugins for the browser. However it may not be powerful enough for what you want in which case you going to need to look at Silver light etc. Could the project not be approached in a more web friendly manor?