Yahoo public api returning result as null. What is the other alternative to get the frre list of stock data live - yahoo-finance

I used below api:
https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22AVGO%22)&format=json&diagnostics=true&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=
It is returning below json:
{
"query": {
"count": 0,
"created": "2017-12-15T10:17:09Z",
"lang": "en-US",
"diagnostics": {
"redirect": [{
"from": "/ec?url=http%3a%2f%2fwww.datatables.org%2fyahoo%2ffinance%2fyahoo.finance.quotes.xml&t=1513333026&ttl=60&sig=5p8s71uMOfbvHH8Cx7iHVQ--~D",
"status": "307",
"content": "http://www.datatables.org/yahoo/finance/yahoo.finance.quotes.xml"
}, {
"from": "/ec?url=http%3a%2f%2fdownload.finance.yahoo.com%2fd%2fquotes.csv%3ff%3daa2bb2b3b4cc1c3c4c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy%26s%3dAVGO&t=1513333029&ttl=60&sig=lMli2lKRDkqtxVXcjK.Vyg--~D",
"status": "307",
"content": "http://download.finance.yahoo.com/d/quotes.csv?f=aa2bb2b3b4cc1c3c4c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy&s=AVGO"
}],
"url": [{
"execution-start-time": "0",
"execution-stop-time": "2",
"execution-time": "2",
"content": "http://www.datatables.org/yahoo/finance/yahoo.finance.quotes.xml"
}, {
"execution-start-time": "2504",
"execution-stop-time": "2509",
"execution-time": "5",
"http-status-code": "403",
"http-status-message": "Forbidden",
"content": "http://download.finance.yahoo.com/d/quotes.csv?f=aa2bb2b3b4cc1c3c4c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy&s=AVGO"
}],
"publiclyCallable": "true",
"cache": {
"error": "Timed out waiting for operation - failing node: memcache3.yql.gq1.yahoo.com/206.190.36.58:11211",
"execution-start-time": "3",
"execution-stop-time": "2503",
"execution-time": "2500",
"method": "GET",
"type": "MEMCACHED",
"content": "5d1e1de680846a307c9874dc3d6878dc"
},
"javascript": ["Unable to retrieve query results from cache, Timed out waiting for operation - failing node: memcache3.yql.gq1.yahoo.com/206.190.36.58:11211", {
"execution-start-time": "2",
"execution-stop-time": "2509",
"execution-time": "2506",
"instructions-used": "12000",
"table-name": "yahoo.finance.quotes"
}],
"csv": "Column mismatch: [Ask, AverageDailyVolume, Bid, AskRealtime, BidRealtime, BookValue, Change&PercentChange, Change, Commission, Currency, ChangeRealtime, AfterHoursChangeRealtime, DividendShare, LastTradeDate, TradeDate, EarningsShare, ErrorIndicationreturnedforsymbolchangedinvalid, EPSEstimateCurrentYear, EPSEstimateNextYear, EPSEstimateNextQuarter, DaysLow, DaysHigh, YearLow, YearHigh, HoldingsGainPercent, AnnualizedGain, HoldingsGain, HoldingsGainPercentRealtime, HoldingsGainRealtime, MoreInfo, OrderBookRealtime, MarketCapitalization, MarketCapRealtime, EBITDA, ChangeFromYearLow, PercentChangeFromYearLow, LastTradeRealtimeWithTime, ChangePercentRealtime, ChangeFromYearHigh, PercebtChangeFromYearHigh, LastTradeWithTime, LastTradePriceOnly, HighLimit, LowLimit, DaysRange, DaysRangeRealtime, FiftydayMovingAverage, TwoHundreddayMovingAverage, ChangeFromTwoHundreddayMovingAverage, PercentChangeFromTwoHundreddayMovingAverage, ChangeFromFiftydayMovingAverage, PercentChangeFromFiftydayMovingAverage, Name, Notes, Open, PreviousClose, PricePaid, ChangeinPercent, PriceSales, PriceBook, ExDividendDate, PERatio, DividendPayDate, PERatioRealtime, PEGRatio, PriceEPSEstimateCurrentYear, PriceEPSEstimateNextYear, Symbol, SharesOwned, ShortRatio, LastTradeTime, TickerTrend, OneyrTargetPrice, Volume, HoldingsValue, HoldingsValueRealtime, YearRange, DaysValueChange, DaysValueChangeRealtime, StockExchange, DividendYield] vs Yahoo! - 403 Forbidden -- error 403It has come to our attention that this service is being used in violation of the Yahoo Terms of Service. As such, the service is being discontinued. For all future markets and equities data research, please refer to finance.yahoo.com.",
"query": {
"execution-start-time": "2503",
"execution-stop-time": "2509",
"execution-time": "6",
"params": "{url=[http://download.finance.yahoo.com/d/quotes.csv?f=aa2bb2b3b4cc1c3c4c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy&s=AVGO]}",
"content": "select * from csv where url=#url and columns='Ask,AverageDailyVolume,Bid,AskRealtime,BidRealtime,BookValue,Change&PercentChange,Change,Commission,Currency,ChangeRealtime,AfterHoursChangeRealtime,DividendShare,LastTradeDate,TradeDate,EarningsShare,ErrorIndicationreturnedforsymbolchangedinvalid,EPSEstimateCurrentYear,EPSEstimateNextYear,EPSEstimateNextQuarter,DaysLow,DaysHigh,YearLow,YearHigh,HoldingsGainPercent,AnnualizedGain,HoldingsGain,HoldingsGainPercentRealtime,HoldingsGainRealtime,MoreInfo,OrderBookRealtime,MarketCapitalization,MarketCapRealtime,EBITDA,ChangeFromYearLow,PercentChangeFromYearLow,LastTradeRealtimeWithTime,ChangePercentRealtime,ChangeFromYearHigh,PercebtChangeFromYearHigh,LastTradeWithTime,LastTradePriceOnly,HighLimit,LowLimit,DaysRange,DaysRangeRealtime,FiftydayMovingAverage,TwoHundreddayMovingAverage,ChangeFromTwoHundreddayMovingAverage,PercentChangeFromTwoHundreddayMovingAverage,ChangeFromFiftydayMovingAverage,PercentChangeFromFiftydayMovingAverage,Name,Notes,Open,PreviousClose,PricePaid,ChangeinPercent,PriceSales,PriceBook,ExDividendDate,PERatio,DividendPayDate,PERatioRealtime,PEGRatio,PriceEPSEstimateCurrentYear,PriceEPSEstimateNextYear,Symbol,SharesOwned,ShortRatio,LastTradeTime,TickerTrend,OneyrTargetPrice,Volume,HoldingsValue,HoldingsValueRealtime,YearRange,DaysValueChange,DaysValueChangeRealtime,StockExchange,DividendYield'"
},
"user-time": "2509",
"service-time": "2507",
"build-version": "2.0.217"
},
"results": null
}
}

Yahoo shut their stock api down.

You can get live data using the yahoo_fin package. Its documentation is here: http://theautomatic.net/yahoo_fin-documentation/. You'll mainly need two functions from it -- get_data, and get_quote_table.
If you want live prices, you can do this:
from yahoo_fin.stock_info import get_data, get_quote_table
# get current price for Apple's stock
get_data("AAPL").close[-1]
Running the above code during the trading day will get you the current live price.
If you want other information, like what's available on the quote page (e.g. https://finance.yahoo.com/quote/AAPL?p=AAPL), you can use the get_quote_table function:
get_quote_table("AAPL")
So you just need to replace "AAPL" with whatever ticker symbols you want.
The yahoo_fin package also has functions to get ticker lists from the S&P, NASDAQ, and Dow -- so if you want to pull all the data for those, you can just loop the ticker lists those functions return.
from yahoo_fin.stock_info import tickers_dow
# get all Dow tickers
dow_tickers = tickers_dow()
# get price data for Dow stocks
dow_data = {ticker : get_data(ticker) for ticker in dow_tickers}

Related

How to find document from CouchDB based on properties of other documents in a single query?

There is already an existing CouchDB database which was created based on existing records from a MySQL database.
I have a set of documents like this:
[
{
"_id": "lf_event_users_1247537_11434",
"_rev": "1-19e90d3f19e9da7cc5adab44ebbe3894",
"TS_create": "2018-12-17T10:29:20",
"emm_id": 204662,
"eu_user_id": 201848611,
"type": "lf_event_users",
"uid": 1247537,
"vendor_id": 11434
},
{
"_id": "lf_event_users_1247538_11434",
"_rev": "1-0d0d1e9f1fb5aad9bafd4c53a6cada17",
"TS_create": "2018-12-17T10:29:20",
"emm_id": 204661,
"eu_user_id": 201848611,
"type": "lf_event_users",
"uid": 1247538,
"vendor_id": 11434
},
{
"_id": "lf_event_users_1247539_11434",
"_rev": "1-09bc2bfc709ee9c6e6cac9cb34964ac4",
"TS_create": "2018-12-17T10:29:20",
"emm_id": 204660,
"eu_user_id": 201848611,
"type": "lf_event_users",
"uid": 1247539,
"vendor_id": 11434
}
]
As you can see, all of them are for the same "eu_user_id" = 201848611, and each one has a different "emm_id".
Now, I have another set of document like this in the same CouchDB database:
[
{
"_id": "lf_event_management_master_204660_11434",
"_rev": "2-320111a3814a3efd6838baa0fb5412bb",
"emm_disabled": "n",
"emm_title": "Scanned for local delivery",
"settings": {
"event_view": "ScannedForLocalDeliveryEvent",
"sort_weight": 0
},
"type": "lf_event_management_master",
"uid": 204660,
"vendor_id": 11434
},
{
"_id": "lf_event_management_master_204661_11434",
"_rev": "2-e6d6ebbd4dc4ca473a376d3d16a58e93",
"emm_disabled": "n",
"emm_title": "Local Delivery Cancelled",
"settings": {
"event_view": "CancelDeliveryEvent",
"sort_weight": 4
},
"type": "lf_event_management_master",
"uid": 204661,
"vendor_id": 11434
},
{
"_id": "lf_event_management_master_204662_11434",
"_rev": "2-53cb3d3eba80704e87ea5ff8d5c269df",
"emm_disabled": "n",
"emm_title": "Local Delivery Exception",
"settings": {
"event_view": "DeliveryExceptionEvent",
"sort_weight": 3
},
"type": "lf_event_management_master",
"uid": 204662,
"vendor_id": 11434
}
]
As you can see, each document in this last set has a "uid" matching the "emm_id" in the previous set of documents. Basically this means:
A "user" has many allowed "events".
You can see also that the documents of type "lf_event_management_master" has no "eu_user_id" value or any other key matching this.
My question is:
How can I get all documents of type "lf_event_management_master" allowed for user "201848611" in a single query?
In my case, I only have the User ID (201848611) available at the point where I need to get the allowed events. Currently what is happening is:
I get all the "lf_event_users" records for this user.
Loop all results from previous query and build a new query, extracting this time to find all the "lf_event_management_master" where the "uid" includes any of the "emm_id" values found with the previous query.
Thank you in advance.

In Watson Discovery, limiting "return"ed fields to aggregation values

For the Discovery REST api, the argument/parameter "return" controls which fields are returned.
So if I pass these arguments to the API
{
"query": named_sector,
"count": "10",
"filter": filter_dates,
"aggregation" : "term(docSentiment.type,count:3)"
}
my_query = discovery.query(my_disc_environment_id, my_disc_collection_id, qopts)
print(json.dumps(my_query, indent=2))
I get the following:
{
"matching_results": 14779,
"aggregations": [
{
"type": "term",
"field": "docSentiment.type",
"count": 3,
"results": [
{
"key": "positive",
"matching_results": 4212
},
{
"key": "negative",
"matching_results": 3259
},
{
"key": "neutral",
"matching_results": 152
}
]
}
],
"results": [
{
"id": "6389715fe7e7f711e0bc09d4f1236639",
"score": 1.3689895,
"yyyymm": "201704",
"url": "https://seekingalpha.com/article/4060446-valuation-dashboard-consumer-discretionary-update",
"enrichedTitle": null,
"host": "seekingalpha.com",
"text": "Valuation Dashboard: Consumer Discretionary - Update\n\nSummary\n\nValuation metrics in Consumer Discretionary.\n\nEvolution since last month.\n\nA list of stocks loo ....
and thousands of more lines. How do I restrict the output to the aggregations section? Is this an issue of me better handling the JSON structure that is returned?
thanks
If you change the count argument to 0, the returned JSON will only contain the aggregations.
Also, if you're using the Discovery web tooling, you can enter 0 for the "Number of results to return (Count)" field.
More details and an example can be found here: https://www.ibm.com/watson/developercloud/doc/discovery/using.html#building-aggregations

CEP generate Measurement/Event not in UTC time but in Local

All Measurements came with a time-stamp (event time) of when the measurement was created. Some of these measurements are artificial ones, meaning that they are not created by the device itself, but by a CEP rule running inside the CoT.
The "normal" measurements have the time format coded as UTC
[{
"id": "12704547",
"data": {
"data": {
"time": "2016-07-25T15:24:11.000Z",
"id": "1152930",
"self": "http://testTenant.c8y.com/measurement/measurements/1152930",
"source": {
"id": "222812",
"self": "http://testTenant.c8y.com/inventory/managedObjects/222812"
},
"type": "tsystems_cumulocity_energymeter_digital_ping",
"Energieverbrauch": {
"Ping": {
"unit": "Wh",
"value": 1
}
}
},
"realtimeAction": "CREATE"
},
"channel": "/measurements/222812"
}, {
"successful": true,
"channel": "/meta/connect"
}]
But the "artificial" measurements (created by the CEP rule) use a timestamp with local time
[{
"id": "12704578",
"data": {
"data": {
"time": "2016-07-25T17:24:00.952+02:00",
"id": "1152931",
"self": "http://testTenant.c8y.com/measurement/measurements/1152931",
"source": {
"id": "222812",
"self": "http://testTenant.c8y.com/inventory/managedObjects/222812"
},
"type": "tsystems_cumulocity_energymeter_power_consumption",
"Leistung": {
"Aggregation_1min": {
"unit": "W",
"value": 900
}
}
},
"realtimeAction": "CREATE"
},
"channel": "/measurements/222812"
}]
The measurements from one device should always be encoded with the same timezone (UTC preferred) as different timezone can create problems in clients using that data.
I create the 'time' in the CEP with
current_timestamp().toDate() as time
please use:
com.cumulocity.model.util.DateTimeUtils.newUTC(current_timestamp().toDate()) as dateTime,
instead of
current_timestamp().toDate() as time
in your cep rule.
Best regards,
Arkadiusz
Cumulocity Support Team

Using YQL to get Options Data

Using the YQL console I did:
select * from yahoo.finance.options where symbol="FB"
but I got the following result in JSON. There is no data in the optionsChain. Based on other posts I believe the request to be formatted correctly. How do I get the data that should be in the options chain? Is this out of date or am I missing something? Thanks in advance.
{
"query": {
"count": 1,
"created": "2015-01-10T17:26:16Z",
"lang": "en-US",
"diagnostics": {
"url": [
{
"execution-start-time": "0",
"execution-stop-time": "71",
"execution-time": "71",
"content": "http://www.datatables.org/yahoo/finance/yahoo.finance.options.xml"
},
{
"execution-start-time": "74",
"execution-stop-time": "1620",
"execution-time": "1546",
"content": "http://finance.yahoo.com/q/op?s=FB"
}
],
"publiclyCallable": "true",
"log": [
"results.length(): 0",
"no elements"
],
"javascript": {
"execution-start-time": "74",
"execution-stop-time": "1647",
"execution-time": "1573",
"instructions-used": "11272",
"table-name": "yahoo.finance.options"
},
"user-time": "1648",
"service-time": "1537",
"build-version": "0.2.278"
},
"results": {
"optionsChain": {
"symbol": "FB"
}
}
}
}
Lots of people having problems with this myself included.
After looking around I found out that Google's Option information is also available via JSON.
It's no good for production environments as it's unsupported and really just a function of how there site operates, but if your interested I did a tutorial with full sample code on it. Even better it's REST based so can be consumed fairly easily in any language.
Google Stock Option Chain Data

Option symbol 'GOOG' not working in YQL

I am trying to retrieve options data from Yahoo Finance using YQL. The very strange problem is that I can download the desired options data for other symbols such as AAPL (Apple) and MSFT (Microsoft), but not for GOOG (Google).
Here's my query and the result. Obviously I can retrieve data for AAPL and MSFT, but GOOG is missing. This happens not only with yahoo.finance.option_contracts, but also with all other option-related tables including yahoo.finance.options. I am pretty sure that this is not something related to the query limits, since the query shown below is expected to retrieve only less than 10 results for GOOG.
select * from yahoo.finance.option_contracts where symbol in ( 'GOOG', 'AAPL', 'MSFT')
{
"query": {
"count": 2,
"created": "2014-03-03T15:00:45Z",
"lang": "en-US",
"diagnostics": {
"url": [
{
"execution-start-time": "0",
"execution-stop-time": "1",
"execution-time": "1",
"content": "http://www.datatables.org/yahoo/finance/yahoo.finance.option_contracts.xml"
},
{
"execution-start-time": "4",
"execution-stop-time": "1057",
"execution-time": "1053",
"content": "http://finance.yahoo.com/q/op?s=MSFT"
},
{
"execution-start-time": "5",
"execution-stop-time": "3571",
"execution-time": "3566",
"content": "http://finance.yahoo.com/q/op?s=AAPL"
},
{
"error": "Connect Failure",
"execution-start-time": "5",
"execution-stop-time": "10017",
"execution-time": "10012",
"content": "http://finance.yahoo.com/q/op?s=GOOG"
},
{
"error": "Connect Failure",
"execution-start-time": "5",
"execution-stop-time": "10017",
"execution-time": "10012",
"content": "http://finance.yahoo.com/q/op?s=GOOG"
}
],
"publiclyCallable": "true",
"javascript": [
{
"execution-start-time": "3",
"execution-stop-time": "1074",
"execution-time": "1070",
"instructions-used": "18571",
"table-name": "yahoo.finance.option_contracts"
},
{
"execution-start-time": "3",
"execution-stop-time": "3616",
"execution-time": "3612",
"instructions-used": "31142",
"table-name": "yahoo.finance.option_contracts"
},
"Exception: Read timed out, url: http://finance.yahoo.com/q/op?s=GOOG",
{
"execution-start-time": "3",
"execution-stop-time": "10017",
"execution-time": "10013",
"instructions-used": "31142",
"table-name": "yahoo.finance.option_contracts"
}
],
"user-time": "10018",
"service-time": "19447",
"build-version": "0.2.2157"
},
"results": {
"option": [
{
"symbol": "AAPL",
"contract": [
"2014-03",
"2014-04",
"2014-05",
"2014-06",
"2014-07",
"2014-10",
"2015-01",
"2016-01"
]
},
{
"symbol": "MSFT",
"contract": [
"2014-03",
"2014-04",
"2014-05",
"2014-06",
"2014-07",
"2014-10",
"2015-01",
"2016-01"
]
}
]
}
}
}
In sum, every time I try to retrieve options data for GOOG, I only get null result with the error of 'Read Timed Out'. For other option symbols such as AAPL and MSFT, I get what I desire.
Anyone knows why this is so?
This must have to do with Google the company trading under TWO stickers (See http://finance.yahoo.com/news/google-trades-under-2-tickers-225300863.html): GOOG and GOOGL.
YQL doesn't recognize GOOG, but will accept the new ticker GOOGL. I suspect the YQL query errors out on getting unexpectedly two tickers instead of one when looking for just GOOG.
If you request a GOOG quote on Yahoo!Finance, it works.

Resources