There is requirement where we have to calculate the throughput for the agile train teams. Is there any chart in JIRA using which we can calculate throughput for agile train teams.
Probably the closest thing to this is the control chart.
It will show you the rolling average of the mean time it takes for items to pass through your team's workflow.
This script might be really useful for you: http://www.littlebluemonkey.com/blog/automatically-import-jira-backlog-into-google-spreadsheet
It's a script that connects a Google Spreadsheet to Jira, bringing all the details related to your cards, so you can easily calculate throughput, lead time, flow efficiency, whatever the metrics you want.
Honestly the data quality in Jira round this can be a problem.
~ With a budget: Actionable Agile makes a great plugin that cleans it up. But there'd be a cost of course. https://www.55degrees.se/actionableagile
~ Without: Troy Megennis of Focused Objective is a writer and speaker who specializes in Coaching With Data. You could try his downloadable dashboard (requires Excel)
https://www.focusedobjective.com/w/support/
It also does Monte Carlo simulations
Recently I tried to build a model to analyze the importance and effect of features of my app.
Features are about user behaviors which includes pages that users have viewed, buttons that users have clicked and so on.
In my linear model, I found that almost all features have positive effects. In the other word, the more active the higher probability for users to stay.
Under this condition, ads have also became a positive feature.
Users who have viewed several ads are more likely to stay than those who haven't viewed any ads just because they are more active on my app.
So I want to know how to design the metrics to evaluate the indeed effect of every feature.
Principal component analysis is what you need. It ranks the features in order of their impact on your model.
This paper can help you to get familiar with it.
The TFS Scrum and Agile templates come with a Velocity report. It includes the story points completed for each sprint and an average velocity. After 27 sprints and various changes to the team, earlier results are no longer relevant to calculating our current velocity.
The average for the entire life of the project is calculated as
=Avg(Fields!Story_Points.Value, "dsVelocity")
How do I calculate the rolling average for the last 6 sprints?
If you try to calculate the average velocity for your team over the last several sprints. It's impossible by now. It will include all sprints.
As a workaround, you may need to calculate this manually. You can also add a feature request in User Voice Site: https://visualstudio.uservoice.com/forums/330519-team-services
If I am build a project applying Lambda-architecture now, should I split the batch layer and the serving layer, i.e. program A do the batch layer's work, program B do the serving layer's? they are physically independent but logically relevant, since program A can tell B to work after A finishes the pre-compute works.
If so, would you please tell me how to implement it? I am thinking about IPC. If IPC could help, what is the specific way?
BTW, what does "batch view" mean exactly? why and How does the serving layer index it?
What is the best way to implement Lambda Architecture batch_layer and serving_layer? That totally depends upon the specific requirements, system environment, and so on. I can address how to design Lambda Architecture batch_layer and serving_layer, though.
Incidentally, I was just discussing this with a colleague yesterday and this is based on that discussion. I will explain in 3 parts and for the sake of this discussion lets say we are interested in designing a system that computes the most read stories (a) of the day, (b) of the week, (c) of the year:
Firstly in a lambda architecture, it is important to divide the problem that you are trying to solve with respect to time first and features second. So if you model your data as an incoming stream then the speed layer deals with the 'head' of the stream, e.g. current day's data; the batch layer deals with the 'head' + 'tail', the masterset.
Secondly, divide the features between these time-based lines. For instance, some features can be done using the 'head' of the stream alone, while other features require a wider breadth of data than the 'head', e.g. masterset. In our example, lets say that we define the speed layer to compute one day's worth of data. Then the Speed layer would compute most read stories (a) of the day in the so-called Speed View; while the Batch Layer would compute most read stories (a) of the days, (b) of the week, and (c) of the year in the so-called Batch View. Note that yes there may appear to be a bit of redundancy but hold on to that thought.
Thirdly, serving layer response to queries from clients regarding Speed View and Batch View and merges results accordingly. There will necessarily be overlap in the results from the Speed View and the Batch View. No matter, this is divide of Speed vs Batch, among other benefits, allows us to minimize exposure to risks such as (1) rolling out bugs, (2) corrupt data delivery, (3) long-running batch processes, etc. Ideally, issues will be caught in the speed view and then fixes will be applied prior to the batch view re-compute. If it is then all is well and good.
In summary, no IPC needs to be used since they are completely independent from each other. So program A does not need to communicate to program B. Instead, the system relies upon some overlap of processing. For instance, if program B computes its Batch view based on a daily basis then program A needs to compute the Speed view for day plus any additional time that processing may take. This extra time needs to include any downtime in the batch layer.
Hope this helps!
Notes:
Redundancy of the batch layer - it is necessary to have at least some redundancy in the batch layer since the serving layer must be able to provide a single cohesive view of results to queries. At the least, the redundancy may help avoid time-gaps in query responses.
Assessing which features are in the speed layer - this step will not always be as convenient as in the 'most read stories' example here. This is more of an art form.
Given a set of data very similar to the Motley Fool CAPS system, where individual users enter BUY and SELL recommendations on various equities. What I would like to do is show each recommendation and I guess some how rate (1-5) as to whether it was good predictor<5> (ie. correlation coefficient = 1) of the future stock price (or eps or whatever) or a horrible predictor (ie. correlation coefficient = -1) or somewhere in between.
Each recommendation is tagged to a particular user, so that can be tracked over time. I can also track market direction (bullish / bearish) based off of something like sp500 price. The components I think that would make sense in the model would be:
user
direction (long/short)
market direction
sector of stock
The thought is that some users are better in bull markets than bear (and vice versa), and some are better at shorts than longs- and then a combination the above. I can automatically tag the market direction and sector (based off the market at the time and the equity being recommended).
The thought is that I could present a series of screens and allow me to rank each individual recommendation by displaying available data absolute, market and sector out performance for a specific time period out. I would follow a detailed list for ranking the stocks so that the ranking is as objective as possible. My assumption is that a single user is right no more than 57% of the time - but who knows.
I could load the system and say "Lets rank the recommendation as a predictor of stock value 90 days forward"; and that would represent a very explicit set of rankings.
NOW here is the crux - I want to create some sort of machine learning algorithm that can identify patterns over a series of time so that as recommendations stream into the application we maintain a ranking of that stock (ie. similar to correlation coefficient) as to the likelihood of that recommendation (in addition to the past series of recommendations ) will affect the price.
Now here is the super crux. I have never taken an AI class / read an AI book / never mind specific to machine learning. So I cam looking for guidance - sample or description of a similar system I could adapt. Place to look for info or any general help. Or even push me in the right direction to get started...
My hope is to implement this with F# and be able to impress my friends with a new skill set in F# with an implementation of machine learning and potentially something (application / source) I can include in a tech portfolio or blog space;
Thank you for any advice in advance.
I have an MBA, and teach data mining at a top grad school.
The term project this year was to predict stock price movements automatically from news reports. One team had 70% accuracy, on a reasonably small sample, which ain't bad.
Regarding your question, a lot of companies have made a lot of money on pair trading (find a pair of assets that normally correlate, and buy/sell pair when they diverge). See the writings of Ed Thorpe, of Beat the Dealer. He's accessible and kinda funny, if not curmudgeonly. He ran a good hedge fund for a long time.
There is probably some room in using data mining to predict companies that will default (be unable to make debt payments) and shorting†them, and use the proceeds to buy shares in companies less likely to default. Look into survival analysis. Search Google Scholar for "predict distress" etc in finance journals.
Also, predicting companies that will lose value after an IPO (and shorting them. edit: Facebook!). There are known biases, in academic literature, that can be exploited.
Also, look into capital structure arbitrage. This is when the value of the stocks in a company suggest one valuation, but the value of the bonds or options suggest another value. Buy the cheap asset, short the expensive one.
Techniques include survival analysis, sequence analysis (Hidden Markov Models, Conditional Random Fields, Sequential Association Rules), and classification/regression.
And for the love of God, please read Fooled By Randomness by Taleb.
†shorting a stock usually involves calling your broker (that you have a good relationship with) and borrowing some shares of a company. Then you sell them to some poor bastard. Wait a while, hopefully the price has gone down, you buy some more of the shares and give them back to your broker.
My Advice to You:
There are several Machine Learning/Artificial Intelligence (ML/AI) branches out there:
http://www-formal.stanford.edu/jmc/whatisai/node2.html
I have only tried genetic programming, but in the "learning from experience" branch you will find neural nets. GP/GA and neural nets seem to be the most commonly explored methodologies for the purpose of stock market predictions, but if you do some data mining on Predict Wall Street, you might be able to utilize a Naive Bayes classifier to do what you're interested in doing.
Spend some time learning about the various ML/AI techniques, get a small data set and try to implement some of those algorithms. Each one will have its strengths and weaknesses, so I would recommend that you try to combine them using Naive Bays classifier (or something similar).
My Experience:
I'm working on the problem for my Masters Thesis so I'll pitch my results using Genetic Programming: www.twitter.com/darwins_finches
I started live trading with real money in 09/09/09.. yes, it was a magical day! I post the GP's predictions before the market opens (i.e. the timestamps on twitter) and I also place the orders before the market opens. The profit for this period has been around 25%, we've consistently beat the Buy & Hold strategy and we're also outperforming the S&P 500 with stocks that are under-performing it.
Some Resources:
Here are some resources that you might want to look into:
Max Dama's blog: http://www.maxdama.com/search/label/Artificial%20Intelligence
My blog: http://mlai-lirik.blogspot.com/
AI Stock Market Forum: http://www.ai-stockmarketforum.com/
Weka is a data mining tool with a collection of ML/AI algorithms: http://www.cs.waikato.ac.nz/ml/weka/
The Chatter:
The general consensus amongst "financial people" is that Artificial Intelligence is a voodoo science, you can't make a computer predict stock prices and you're sure to loose your money if you try doing it. None-the-less, the same people will tell you that just about the only way to make money on the stock market is to build and improve on your own trading strategy and follow it closely.
The idea of AI algorithms is not to build Chip and let him trade for you, but to automate the process of creating strategies.
Fun Facts:
RE: monkeys can pick better than most experts
Apparently rats are pretty good too!
I understand monkeys can pick better than most experts, so why not an AI? Just make it random and call it an "advanced simian Mersenne twister AI" or something.
Much more money is made by the sellers of "money-making" systems then by the users of those systems.
Instead of trying to predict the performance of companies over which you have no control, form a company yourself and fill some need by offering a product or service (yes, your product might be a stock-predicting program, but something a little less theoretical is probably a better idea). Work hard, and your company's own value will rise much quicker than any gambling you'd do on stocks. You'll also have plenty of opportunities to apply programming skills to the myriad of internal requirements your own company will have.
If you want to go down this long, dark, lonesome road of trying to pick stocks you may want to look into data mining techniques using advanced data mining software such as SPSS or SAS or one of the dozen others.
You'll probably want to use a combination or technical indicators and fundamental data. The data will more than likely be highly correlated so a feature reduction technique such as PCA will be needed to reduce the number of features.
Also keep in mind your data will constantly have to be updated, trimmed, shuffled around because market conditions will constantly be changing.
I've done research with this for a grad level class and basically I was somewhat successful at picking whether a stock would go up or down the next day but the number of stocks in my data set was fairly small (200) and it was over a very short time frame with consistent market conditions.
What I'm trying to say is what you want to code has been done in very advanced ways in software that already exists. You should be able to input your data into one of these programs and using either regression, or decision trees or clustering be able to do what you want to do.
I have been thinking of this for a few months.
I am thinking about Random Matrix Theory/Wigner's distribution.
I am also thinking of Kohonen self-learning maps.
These comments on speculation and past performance apply to you as well.
I recently completed my masters thesis on deep learning and stock price forecasting. Basically, the current approach seems to be LSTM and other deep learning models. There are also 10-12 technical indicators (TIs) based on moving average that have been shown to be highly predictive for stock prices, especially indexes such as SP500, NASDAQ, DJI, etc. In fact, there are libraries such as pandas_ta for computing various TIs.
I represent a group of academics that are trying to predict stocks in a general form that can also be applied to anything, even the rating of content.
Our algorithm, which we describe as truth seeking, works as follows.
Basically each participant has their own credence rating. This means that the higher your credence or credibility, then the more their vote counts. Credence is worked out by how close to the weighted credence each vote is. It's like you get a better credence value the closer you get to the average vote that has already been adjusted for credence.
For example, let's say that everyone is predicting that a stock's value will be at value X in 30 day's time (a future's option). People who predict on the average get a better credence. The key here is that the individual doesn't know what the average is, only the system. The system is tweaked further by weighting the guesses so that the target spot that generates the best credence is those votes that are already endowed with more credence. So the smartest people (historically accurate) project the sweet spot that will be used for further defining who gets more credence.
The system can be improved too to adjust over time. For example, when you find out the actual value, those people who guessed it can be rewarded with a higher credence. In cases where you can't know the future outcome, you can still account if the average weighted credence changes in the future. People can be rewarded even more if they spotted the trend early. The point is we don't need to even know the outcome in the future, just the fact that the weighted rating changed in the future is enough to reward people who betted early on the sweet spot.
Such a system can be used to rate anything from stock prices, currency exchange rates or even content itself.
One such implementation asks people to vote with two parameters. One is their actual vote and the other is an assurity percentage, which basically means how much a particular participant is assured or confident of their vote. In this way, a person with a high credence does not need to risk downgrading their credence when they are not sure of their bet, but at the same time, the bet can be incorporated, it just won't sway the sweet spot as much if a low assurity is used. In the same vein, if the guess is directly on the sweet spot, with a low assurity, they won't gain the benefits as they would have if they had used a high assurity.