We have 1 million products. We are using “The Ajax Filter Pro Pack” wherein combination we are using is “Category & Manufactures and Specification”
The challenge is we are facing a huge wait time of 15 seconds to filter and display the products.
We need to resolve it and bring it down to 2 seconds as its effecting our customer experience badly.
Related
I'm developing an app that uses CloudKit as it's main database. I have a relatively small database (around 200 entries, each with 2-3 relationships) that I offer pre-populated as as the public database (new in iOS 14).
I noticed that the CK mirroring is very very slow. I get the first 5-6 transactions in a matter of seconds an that I have to wait around 2 minutes for all the data to populate.
As I can't show partial results (I can't allow the user to see the main entity if it's relations are not fetched yet) this is a big problem for me.
Is there a way to speed up the CK mirroring process? (make it more efficient)?
How can I diagnose what is taking that much? Apple recommended in the last WWDC to use this public database as an initial set of data, but people will get frustrated if the initial app load takes 2 minutes :o
This is kinda expected behaviour, lets say for example you have 500 entries without any relationships and you want to fetch then with CKQueryOperation, when the operation is added to the public container it will not return all of the 500 entries at once, it will return at least 100 then it will query for the next 100 using the cursor and so on.
Edit
Is there a way to speed up the CK mirroring process? (make it more
efficient)?
No
These operations are maybe slow because of network issues, you should take that in consideration.
I want to retrieve all stocks from few exchanges - by retrieve the stocks that inside those exchanges (by taking from http://www.nasdaq.com/screening/company-list.aspx).
And then I will quote for all stocks from google or Yahoo.
My question is if I will quote all of them for every 5 seconds or 10 seconds - will they block me?
What is the correct way for getting all stocks and they updated data?
Thanks!
David,
tl;dr - yahoo finace is OK (scraping 2,000 stocks) if you insert pauses in your code
I have some clumsy, but working code (my first attempt at scrapping) that pulls some data from Yahoo Finance. While I don't like the code and I will rewrite it for nasdaq.com in following weeks, I can tell you that I'm not getting blocked.
I have a few years old list of stocks for Russel 2000 so there are around 2,000 tickers I'm slowly going through and pulling some data from balance sheet. I'm using Selenium (see my question history, there is only one to see/get working code), code loads Chromium web browser (Linux) clicks on Balance sheet, scrape some data, clicks quarterly link, scraps more data and then closes the browser. For every ticker (stock).
Just to be on a safe side, I put several pauses into my code, for every scrap or navigation on site I added between 5 and 10 seconds. That way I'm slowly scraping data and Yahoo seems to be OK with this :-) It takes about one minute per ticker. I'm running this scrap job (for the first time!) now for over 30 hours lol and I'm currently at ticker that starts with T so I have few more hours to go.
I have read somewhere that some sites can spot this slow scraping also. So as an idea, instead of just hard code pause of say 7 seconds, you could run random number generator between IDK, 7-15 seconds and that way pauses will be more random and less prone to be spotted... Just a though Hope this helps a little bit even if with delay.
Ah, and if this answer does help you, please be so kind to mark it as solved and up vote it. Maybe I can get a point or two for it. My points are so low I can't even vote other posts that I like and that helped me.
I have an application where I need to get complete, realtime search results from twitter (preferably polling every 500ms or less). Based on my understanding, doing this using the search API will run into rate limits very quickly. However, the streaming API doesn't seem to support getting complete anything (only a 5% sample).
More specifically, I have a search query term which typically comes up with <20 matching tweets per hour, and I would like to be informed of these new tweets within 1-2 seconds, and it is considered a failure if I am not notified within 5 seconds. Due to the relatively low frequency of posting, missing even one tweet is very undesirable.
Is there any way I can realistically do this using twitter API, or is my only choice to write a browser extension to repeatedly refresh the search page?
The answer is "yes". Although you are rate limited (the limit is closer to 1% than 5%), that is only a cutoff based on your query results. Very roughly, you can stream about 60 tweets per second max. In your case, you say you expect under 20 tweets per hour, so you should have no problem getting all those tweets.
You also require a latency less than 5 seconds. In my experience latency has always been a second or two. I think you should be fine.
I have an MVC 4 app which is performing a slow request roughly every 45 seconds. It's normally 200ms and all requests will be 200ms for about 45 seconds before there is one for about 4500ms. Then back to 200ms.
I have no caching set up.
I am however using a PrincipalContext to query AD.
I was wondering if anyone had any idea before I start setting up some profiling on the calls.
quick:
slow:
Thanks Pinch, but the problem was Active Domain and PrincipalContext. See the solution here, I turned off NetBIOS over TCP/IP and everything is running much better now.
Though I was displaying menu items based on the roles a user has, perhaps I'll cache these somewhere so I don't have to query AD every request.
Thanks in advance for the assistance. I recently ran into an issue with a clients app using Flurry Analytics. There is a maximum number of 300 events that can be tracked (each event can have a total of 10 different paramaters) I need to track more then 300 events for this client.
Does Google Analytics have a maximum number of events that can be tracked?
(the reason for the high number of tracked events for those who are curious is because the client has a database of about 15000 products, and wants to know how users are interacting with the products, additionally they want to track the search terms that users are searching for, thus the need for a HUGE number of events to be tracked....)
There is a limit of 10 million hits (could be events, page views etc) per month. Read specifics at https://developers.google.com/analytics/devguides/collection/gajs/limits-quotas. This is the free version. The Premium version supports higher data limits. https://www.google.com/analytics/360-suite/#?modal_active=none
There is also a 10 event (_trackEvent) limit over a 5 second period.
Thought it might be helpful to clarify, considering the changes that Google made in Oct '12.
https://developers.google.com/analytics/devguides/collection/other/limits-quotas
10 million hits per month per web property - If you go over this limit, the Google Analytics team might contact you and ask you upgrade to Premium or implement client sampling to reduce the amount of data being sent to Google Analytics.
...
ga.js or Legacy Libraries
This applies to ga.js, mobile snippets and any other legacy tracking library.
500 hits per session not including ecommerce (item and transaction hit types)
If you go over this limit, additional hits will not be processed for that session.
...
ga.js
Each web property starts with 10 hits that are replenished at 1 hit per second. Applies only to event type hits.
analytics.js
Each web property starts with 20 hits that are replenished at 2 hit per second. Applies to All hits.
Android SDK
For each tracker instance on a device, each app instance starts with 60 hits that are replenished at 1 hit every 2 seconds. Applies to All hits.
iOS SDK
Each property starts with 60 hits that are replenished at 1 hit every 2 seconds. Applies to All hits.