How can I get finance gata from yahoo.finance in realtime? I tried select * from yahoo.finance.historicaldata but there only data by day not by hours.
You're querying a table called historicaldata, it is probably only updated once a day. Another table, quotes, may have data updated on a more frequent timer (hourly, as you've mentioned).
Unfortunately, I think you've run into a limitation of Yahoo's free financial data feeds. They have to purchase market feeds from the source (NASDAQ, NYSE, etc.) (or close to it) and those feeds aren't cheap. If you want truly real-time data, that's something you'll probably have to cough good money out for.
There are other sources besides Yahoo though that might get you down to 15 minute updates. I will leave that research as an exercise for you since that's out of scope for StackOverflow. If you do find a different source and are having trouble reading from it, do please feel free to ask another question here! The more closely related to actual programming the better response you'll get from the community.
Related
I'm interested in using the Tweets in a Complex Networks research project for which I've to collect a good amount of tweets(unbiased sample of tweets) that're related to a particular event, for example, say Boston Bombing. I need to collect the tweets that're related to this event from the event date to current date.
Tried Search API and got to know that I can't retrieve tweets older than a week. I've a small doubt regarding Streaming API, does Streaming API allow you to get older tweets and not real-time tweets. Also, I don't need tweets that're of a particular user.
Is there any way that I can collect the necessary tweets? If there aren't any, can you give me any other archiving sources that I can use?
Thanks a lot.
The streaming API is for accessing what is happening right now, there is no option to "replay" the stream from some arbitrary point in the past. You can access past tweets from individual users' timelines (up to 3,200), but this would require knowing who had tweeted about your event and, as you say, you can only search the past week.
If you are prepared to pay for it, you can get tweets since the start of 2010 via DataSift.
Historical data going back to the very first Tweet ever is only available from Gnip.
Can anyone provide how to get historical options data with strikes by Google Finance API? Mbe Yahoo API can do it?
Thx.
AFAIK, there's no free API that lets you query for historical option prices easily.
Your best shot may be to collect it daily for the stocks you're interested in:
http://www.google.com/finance/option_chain?q=AAPL&output=json
I've never seen free historical option data prices. Neither Yahoo or Google provide them. I downloaded 2012 end of day data from optiondata.net. It includes daily prices, greeks and IV. As far as I can tell it seems accurate. There are several other paid sources out there but I have not tested their data.
I'd like to get a big list (say 1,000 or more) of word phrases that people search for on the internet recently (anything from the most recent month or week or day is ok). Results from Google or any of the bigger search sites would be okay. And is there a way to do this programmatically? Python would be first choice, shell scripts works too. Thanks!
Bonus points for historical results too.
http://www.google.com/trends
google is pretty data friendly
they even provide rss feeds
http://www.google.com/trends/hottrends/atom/hourly
Yes, It's python friendly with API and easy_install to boot!
http://pypi.python.org/pypi/pyGTrends/0.81
Along with what TelsaBoil post, Google Insights looks to give historical results too
http://www.google.com/insights/search/
I think you should check this ones:
http://www.google.com/trends/hottrends
http://www.google.com/trends/hottrends/atom/hourly [RSS Hourly Feed]
http://pypi.python.org/pypi/pyGTrends/0.81 [Python Google Trends Information Retrieval]
I plan to write a thesis about using sentiment information to enhance the predictivity of some financial trading model for currency.
The sentiment data should be twitter threads including some keyword, like "EUR.USD". And I will filter out some sentiment words to identify the sentiment. Simple idea. Then we try to see whether here is any relation between the degree of sentiment and the movement of EUR.USD.
My big concern is on twitter data. As we all know that the twitter set up the limit to see the history data. You could only browser back for like 5 days. It is not enough since our strategy based on daily sentiment.
I noticed that google have some fantastic thing like timeline about the twitter updates: http://www.readwriteweb.com/archives/googles_twitter_timeline_lets_you_explore_the_past.php
But first of all, I am in Switzerland and seems I have no such function on my google which is too smart to identify my location and may block some US google version function like this. Secondly, even I could see some fancy interactive google timeline control on my firefox, How could I dig out data from my query and save them? Does google supply such api?
The Google service you mentioned has shut down recently so you won't be able to use it. (http://www.searchenginejournal.com/google-realtime-shuts-down-as-twitter-deal-expires/31007/)
If you need a longer timespan of data to analyze I see the following options:
pay for historical data :) (https://dev.twitter.com/docs/twitter-data-providers)
if you don't want to pay, you need to fetch tweets containing EUR/USD whatever else (you could use the streaming API for this) and store them somehow. Run this service for a while (if possible) and you'll have more than just 5 days of data.
I'm trying to write a script that would download currency rates from yahoo finance. The problem is ... i can't find any information on the limitations of this service. Especially i'm interested in how often i can query yahoo finance to access the quotes.csv file. Would yahoo kill my script if i executed it periodically every minute or so? Does anyone know where i could find some official yahoo information of things like that? I've been searching for hours, but it's either well hidden or it's just hiding in plain sight and i don't see it...
Usually its against the TOS of the website. However if you want to collect data that way on a small scale it is fairly trivial. I have mined yahoo finance in the past and have never been turned off. Don't hammer the site...space out your requests. If you want to be even more clever about it script a web browser to do it for you. The page requests will then look identical.