I have users in my app, and to put it simple, each user has many pictures. These pictures are viewed somewhere else - not on my website (my website is only for managing them - creating, deleting, showing views stats etc). On the other websites they are shown simply like so:
<img src="http://mywebsite.com/mypicture_132.png" />
And each time someone opens the image I need to gather some analytics, like:
- http referrer
- IP of the viewer
- time the picture was viewed
And then show it to the user on my website.
I was doing it all on my side (by simply saving every image view it to database), but now the servise is too big to maintain this data (that's like hundreds of records per day for every picture, times 50-70 pics per user, lots of data). So I'm thinking to find some powerful external service that can do that for me.
Maybe something like http://goo.gl (as it has pretty good stats per day/week/month/year, charts, everything is stored on Google servers, etc). But that one should be attached to gmail account, and I also don't think it supports simple export of that data. I also need to be able to get total stats for selected pictures, so that user doesn't have to click on each of them individually. Main thing is I want someone else to record and store the data. Don't mind paying a reasonable price.
Do you know any services like this?
It turned out that Google has a Url Shortener API that does just what I need.
Related
I would like to monitor users' page visits and clicks in my Rails app to make recommendations. My questions are:
Is there a Rails gem for this, or Google Analytics is the standard? If latter is true, then how should I link a page visit to a particular user profile?
It is typical in Rails to have a section in application.html.erb, which is shared for all pages. If I add Google Analytics pageview tracking code to in application.html.erb, will it be able to track all individual pages?
There are other ways, but the vast majority probably use Google Analytics. Several gems exist that help you integrate with GA to get at the data. See here: https://www.ruby-toolbox.com/categories/Web_Analytics.
Based on your first question, it seems you may want more insight than GA can provide. I've used ClickTale (http://www.clicktale.com) and Woopra (http://www.woopra.com) before, to good effect. This article lists several other alternatives, too - notice the high marks for Clicky: http://imimpact.com/web-stats-alternatives-to-google-analytics/.
Google Analytics (and almost all of these others) will take care of your second question automatically whenever the user loads a new page, since it keyed by URL. That means that, although you put the GA script code in a single place, each unique page is tracked individually.
If you have AJAX requests that change that page without changing the URL, you'll need to dig in to the GA script API. Essentially you'll need to push a new url (possibly with a # in it) whenever you want to track an AJAX-driven link/button click. See here: http://davidwalsh.name/ajax-analytics
I am biased, but I would recommend checking out impressionist, if you need to integrate the page views into the app in real-time. With analytics you will always have some lag time and you are also relying on an external dependency. Impressionist is good if you need this kind of control, but if you are just looking for simple metrics and don't need to pull them into the app, then analytics is probably the way to go.
Check out Ahoy, at https://github.com/ankane/ahoy. With just a few lines of code in your app, you can track page views and tie them to user accounts.
You can further customize Ahoy to track custom events, both the client (with JavaScript) and server.
Ahoy does not depend on any third-party services.
Currently I'm using Facebook connect and Twitter oauth to authenticate users. and I'm using their profile image urls directly instead of storing them on my server. I guess it's better to save the thumbnails and think eventually may need to do that, but for now am trying to save as much money as possible. (I'm currently using Heroku to host my project and can't store files unless I use S3 or the likes, which would cost me money)
I'm wondering if this is OK, and if most web apps start out this way. Or if I need to store the images.
Also, now I want to let users set up accounts without Twitter or Facebook. In this case, is there a way to use a "third party thumbnail hosting site" (I don't even know if such a thing exists) for free instead of using S3? It can even be a short term hack, since I plan to eventually implement image storage if this gets traction.
I think not storing them is the best way. If someone changes their picture, you always show the latest this way, if you stored them, well then you would have an old version of their picture after time which could make your app look stale.
As for where to store alternatives, I would really consider s3. It is very cheap storage, and thumbnails won't need a lot of space. It would only get expensive if you had a LOT of users, and if you have a LOT of users, you should be able to afford the storage.
Also, have you looked at Gravatar? A lot of people have those linked to their email addresses, it might be another good way to get their picture. http://en.gravatar.com/site/implement/
I'm building a portal that lists certain products and automatically gets the prices from product pages of listed vendors. To get the URL for the product page on a vendor's website, I've been using Google search API and it's been working great - the first result is invariably the page of the product. However, now I'm getting errors saying that Google has blocked my website (actually my develoment machine's IP) from the API because I've been making automated requests such as scraping (the only item that applies).
Fine, Google can go jump off a cliff, but... how do product portals generally get URLs for thewir products? I can enter the URLs manually but that can be a problem if the vendor's website changes the URL scheme somehow. I obviously need an automated way to do this.
I'm making no more than 50-60 requests per day so I don't get what Google wants. Do they want money?
First, they want you to use one of their APIs, not scrape their web page directly. Their custom search API is documented here. Once you register they'll give you an API key. You can get results in JSON format by requesting
https://www.googleapis.com/customsearch/v1?q=SEARCH_TERMS&key=YOUR_KEY
Second, they do like money, but you might be okay. You're allowed 100 searches per day for free; beyond that you're you're going to be charged $5 per thousand searches.
I'm looking for a solution to a particular query. I've got a website that I have made for a client and I coded my own image uploader/thumbnail creator but it's not handling what he wants very well.
So basically I'm looking for alternatives, and someone suggested using Wordpress and one its plugins to help upload albums. This seems straight forward after a bit of reading. But can I integrate it with my current set up:
A user logs into their account on the site (I've set up user accounts in a DB),
They see which albums are associated with this account.
They open the album and the images appear.
So I guess what I'm asking is, how configurable is wordpress to fit in with my current set up. I need to be able to only display albums related to the logged in user to them.
Tbh if my current way worked consistently (the upload/thumbnail creation is all done in PHP), I would prefer to use that, but this has gone on for a while and I just want to give him a clean solution.
You can consider to use this wordpress plugin:
http://wordpress.org/extend/plugins/nextgen-gallery/
Recently search engines have been able to page dynamic content on social networking sites. I would like to understand how this is done. Are there static pages created by a site like Facebook that update semi frequently. Does Google attempt to store every possible user name?
As I understand it, a page like www.facebook.com/username, is not an actual file stored on disk but is shorthand for a query like: select username from users and display the information on the page. How does Google know about every user, this gets even more complicated when things like tweets are involved.
EDIT: I guess I didn't really ask what I wanted to know about. Do I need to be as big as twitter or facebook in order for google to make special ways to crawl my site? Will google automatically find my users profiles if I allow anyone to view them? If not what do I have to do to make that work?
In the case of tweets in particular, Google isn't 'crawling' for them in the traditional sense; they've integrated with Twitter to provide the search results in real-time.
In the more general case of your question, dynamic content is not new to Facebook or Twitter, though it may seem to be. Google crawls a URL; the URL provides HTML data; Google indexes it. Whether it's a dynamic query that's rendering the page, or whether it's a cache of static HTML, makes little difference to the indexing process in theory. In practice, there's a lot more to it (see Michael B's comment below.)
And see Vartec's succinct post on how Google might find all those public Facebook profiles without actually logging in and poking around FB.
OK, that was vastly oversimplified, but let's see what else people have to say..
As far as I know Google isn't able to read and store the actual contents of profiles, because the Google bot doesn't have a Facebook account, and it would be a huge privacy breach.
The bot works by hitting facebook.com and then following every link it can find. Whatever content it sees on the page it hits, it stores. So even if it follows a dynamic url like www.facebook.com/username, it will just remember whatever it saw when it went there. Hopefully in that particular case, it isn't all the private data of said user.
Additionally, facebook can and does provide special instructions that search bots can follow, so that google results don't include a bunch of login pages.
profiles can be linked from outside;
site may provide sitemap