I'm trying to build a ANN model to predict oil prices and I have diffrent economic features, each one as a time serie.
I pretend to predict the monthly prices, but there are some features that are only avaliable in a annual period. Is it possible to use anyway these features to predict the monthly prices or only can use the monthly features?
Thank you an regards,
Gonzalma.
Related
I would like to ask you some suggestions about a time series prediction problem. In particular, I have to predict on a daily basis the total water demand in a certain area, creating a model based on 4 CVSs files containing:
water demand in aggregated form (time series with daily granularity, 2 years data)
amount of water entering the area's cistern (time series with daily granularity, 2 years data)
amount of water leaving the area's cistern (time series with daily granularity, 2 years data)
water request from 4,000 measurements points across the area (time series with daily granularity, 2 years data).
In your opinion, what is the best model for having a good prediction of the water demand in the area, using the available data and features? I can only think of LSTMs or MLP, I don't know if something like ARIMA or (SARIMA) could be useful in this case, seeing that I have many features but not many days.
Thank you in advance for you help :)
Forecasting is inevitably a domain-specific problem because you can often make better decisions about model and methods when you know something about the system or process you are trying to forecast.
There are quite a few academic papers on forecasting domestic water demand which you could look at if you have access:
E.g.
Demand Forecasting for Water Distribution Systems by Chen and Boccelli (2014)
Urban Water Demand Forecasting: Review of Methods and Models by Donkor et al (2014)
Predicting water demand: a review of the methods employed and future possibilities by de Souza Groppo et. al (2019)
I'm not an expert in this domain so you should probably wait for someone who is to answer the question but I think using an auto-regressive model (e.g. ARIMA), as you have suggested, is a good start because demand is essentially due to aggregate human activity which is inherently driven by daily / weekly routines, and seasonal effects.
There are various routines to fit such models to data. Jason Brownlee has a nice tutorial here using Python's statsmodels.tsa package.
You could also see what people have used for residential energy consumption forecasting as the problem is probably very similar to water demand forecasting.
I have built a churn prediction model for a e-commerce company data. In the model, churn criterion is to be inactive for 12 months from the last available date in the data. While building the model, I created some calculated features to consider the activity in the prediction. I added last 3 and 6 months activities of the customers as binary. Their correlation with the churn is 0.5 and 0.7 respectively. When I checked the other churn prediction models on the web, I saw similar metrics in some projects and some others do not include such a metric.
My models' accuracy is around 90% and I am concerned that if I am doing it wrong by putting the last 3 and/or 6 month activities of customers as an input to the model. Moreover, should I be worried about the correlation between 3m activity and 6m activity? I used PCA for the feature extraction keeping the 0.95 of the variance but is it enough to avoid the correlation problem?
I´m working on a machine learning project where I try to predict, what clients will buy a specific product (buying the product is my target variable). I have plenty of features about the clients and enough historical data.
My issue is that my target variable is highly seasonal – most of the product is sold in December, other months have only little sales.
What do I have to do, to compensate this imbalance? Does the target variable need some adjustments? I need the model to have consistent performance across all months. Thanks
The simplest option would be to include month as a feature in some way. Some options for doing that:
One-hot-encode month. Pros: very simple, leads to rather sparse features set
Create a naive-bayes type feature encoding the prior probability of a sale in the given month. e.g. if 60% of sales are in December and uniformly 3.6% of sales in every other month, then this feature would have a value of 0.6 for every sale in December and 0.036 for sales in other months
For both of these methods you would want to ensure you have training data from a full 12 month period and separate evaluation set also from a full 12 month period.
When using AWS Forecast, is there some way to augment our model with "partial future information" in order to improve forecasts?
I have been getting quite solid looking predictions from AWS Forecast so far, but suspect that I could improve the predictions somewhat substantially if I could provide some information about known future events.
I'm very new to forecasting and machine learning and by "partial future information", I mean:
I am trying to predict how the time-series of variable X will behave in the future
I am training a model with past time-series information for many different variables, including X
I would like to also provide known future time-series information for a subset of these variables because 1) they should have a significant impact on predictions and 2) this would give me the ability to perform "what-if" analysis
To be more concrete:
I am trying to predict future revenue from past revenue, web traffic volume, advertising spending, and promotional discounts
AWS Forecast has been providing me with good forecasts so far (I hold back so many months of known data from the model and its predictions about the "future" match the known data quite well)
However, I would really like to also tell AWS Forecast about, for example, a significant advertising campaign that is planned for the near future
I would also really like to be able to vary some future variable or variables and see how they affect the outcome ("what if I spend $Z on advertising next month?")
Currently, I am providing all of our past revenue, web traffic volume, advertising spending, and promotional discount information to AWS Forecast as a "Target Time Series" in the format of a single CSV file with 3 columns (metric name, timestamp, metric value); approximately 15 distinct values of metric name; and about 10,000 total rows of data (several years worth of daily values of 15 variables = ~ 2 * 365 * 15 = ~ 11,000 rows). Every metric is provided over the same time interval (for instance, all of the metrics are provided between 2017-10-01 and 2019-11-25).
I'd like to provide some additional, partial data that highlights known future significant events (spending on advertising, promotional discounts) to improve our predictions even further.
For example:
Revenue from 2017-10-01 to 2019-11-25
Web traffic from 2017-10-01 to 2019-11-25
Ad spend from 2017-10-01 to 2019-11-25
Promotional discounts from 2017-10-01 to 2019-11-25
plus planned ad spend for 2019-11-26 to 2020-02-01
Can someone please help me with some of the terminology and the "how-to" mechanics of this?
In general, to use a variable in your historical data, you need a forecast of it in the future as well. It would be like trying to forecast electrical usage and then putting historical temperatures in the data set. If you don't have a forecast of the future temperatures, that information hasn't done you any good in improving your forecast. Because now I know what the effect of "an extra one degree of temperature on electrical usage", but ¿what do I do with that if I have no idea what the temperature will be tomorrow?
In your case you have 1 metric you want to forecast (revenue) and three supporting pieces of data: traffic, ad spend, discount. It's great that you have future ad spend, but without the other two, you're a bit out of luck (per the prior paragraph).
However, you can still do something here, but you'll just have to make some assumptions. What I would do is choose a fixed value for all dates in the future and set that for all future dates. Perhaps appropriate values would be discount at zero (full price item) and web traffic at—I'm making this up—1K per day. Now you have full data sets for past and future.
With that set up you could now answer the question, albeit with a caveat. The forecast you get out is now saying...
Here's how much revenue we can expect given our planned ad spend, if we offer no discounts and we get 1K people to the website every day.
Perhaps you could improve that by inputting traffic values in the future that are the same from a year prior. In which case, you could now say ...
Here's how much revenue we can expect given our planned ad spend, if we offer no discounts and the website gets the same traffic as this time last year.
You can take that to variations such as "traffic goes up 10%" or you can take a guess at what the discounts will be or, like before, you could replicate your discounts and traffic from a year prior and say...
Here's how much revenue we can expect given our planned ad spend, if we offer discounts just like last year and see website traffic just like last year.
I suspect you get the idea, so I'll stop all the variations. These are, of course, really just future forecasts of those data; however, it's worth nothing that "creating a forecast" of discounts or web-traffic, doesn't have to be complicated and fancy. "The same as last year" is a perfectly valid "forecast" of what's to come.
I am working on an automated tool where it have to support all kinds of data. Without looking at the data, are there any methods to identify that the dataset following time series? Are there any statistical tests?
Plot the data set if you find data points are equally spaced then your data is time series data.
Ex:continuous monitoring of a person’s heart rate, hourly readings of air temperature, daily closing price of a company stock, monthly rainfall data, and yearly sales figures.