I'm new to raster package in R, I was trying to open a .nc file with the package raster and some error popped out. In case you want to try I was using this dataset from copernicus of monthly sea salinity for the years 2018 and 2019 (the grid was Quebec St.Laurence stuary and Gaspesie coast).
I opened similar data files before but never got this error, and a search online did not clarify too much
Here is my script
library(raster)
library(ncdf4)
#Load the .nc files describing SSS.
SSS = stack('SSS.nc')
and the output error
Warning message:
In .rasterObjectFromCDF(x, type = objecttype, band = band, ...) :
"level" set to 1 (there are 17 levels)
thnx
I expected to create a a rasterstack object to work with
There is no error, there is a warning. If you want another level you can select it.
Either way, the "raster" package is obsolete, and you should try this with terra.
You can probably do
library(terra)
x <- rast('SSS.nc')
"Probably" because you do not provide a file. You should at least include a hyperlink to the file you are using. It is hard to help you without your example being reproducible.
Related
I'm still learning to use caret package through The caret Package by Max Kuhn and got stuck in 16.2 Partial Least Squares Discriminant Analysis section while trying to predict using the plsBayesFit model through predict(plsBayesFit, head(testing), type = "prob") as shown in the book as well.
The data used is data(Sonar) from mlbench package, with the data being split as:
inTrain <- createDataPartition(Sonar$Class, p = 2/3, list = FALSE)
sonarTrain <- Sonar[ inTrain, -ncol(Sonar)]
sonarTest <- Sonar[-inTrain, -ncol(Sonar)]
trainClass <- Sonar[ inTrain, "Class"]
testClass <- Sonar[-inTrain, "Class"]
and then preprocessed as follows:
centerScale <- preProcess(sonarTrain)
centerScale
training <- predict(centerScale, sonarTrain)
testing <- predict(centerScale, sonarTest)
after this the model is trained using plsBayesFit <- plsda(training, trainClass, ncomp = 20, probMethod = "Bayes"), followed by predicted using predict(plsBayesFit, head(testing), type = "prob").
When I'm trying to do this I get the following error:
Error in predict.NaiveBayes(object$probModel[[ncomp[i]]], as.data.frame(tmpPred[, : Not all variable names used in object found in newdata
I've checked both the training and testing sets to check for any missing variable but there isn't any. I've also tried to predict using the 2.7.1 version of pls package which was used to render the book at that time but that too is giving me same error. What's happening?
I've tried to replicate your problem using different models, as I have encountered this error as well, but I failed; and caret seems to behave differently now from when I used it.
In any case stumbled upon this Github-issues here, and it seems like that there is a specific problem with the klaR-package. So my guess is that this is simply a bug - and nothing that can be readily fixed here!
A program to run a Schmid-Leiman transformation using SPSS's Matrix language was published in 2005 by Woolf & Preising in Behavior Research Methods volume 37, pages 48 to 58). It is probably not important for you to know what a Schmid-Leiman transformation is, but I'll explain in comments if you feel it is necessary.
In modifying the program for my own data, I'm getting an error I can't figure out:
Error # 12302 in column 12. Text: ,
Syntax error.
Execution of this command stops.
Error in RIGHT HAND SIDE of COMPUTE command.
The MATRIX statement skipped.
Here is the beginning of the code. The error is showing as coming in Line 6:
* Encoding: UTF-8.
* Schmid-Leiman Solution for 2 level higher-order Factor analysis.
Matrix.
* ENTER YOUR SPECIFICATIONS HERE.
* Enter first-order pattern matrix.
Compute F1={.461, .253, -.058, -.069;
.241, .600, .143, .033;
.582, .047, -.077, -.125;
.327, .297, -.120, -.166;
.176, .448, -.240, -.099;
.680, .069, -.036, -.138;
.415, .228, -.091, -.153;
.
.
.
.390, .205, .002, -.098;
.164, .369, -.170, -.047
}.
As shown above, the text generating the error is shown as a comma (,), but the actual text (following the COMPUTE statement) in column 12 is an open bracket ({). So I have no idea what is going on. Can someone help?
For reference, the original code as proposed by Woolf & Preising (2005) is found here;
The Woolf & Preising article is found here
PS: The sample program given in the link above does run on my copy of SPSS. Here's the beginning of that code:
* Schmid-Leiman Solution for 2 level higher-order Factor analysis.
Matrix.
* ENTER YOUR SPECIFICATIONS HERE.
* Enter first-order pattern matrix.
Compute F1={0.099, 0.5647, -0.1521;
0.0124, 0.9419, -0.1535;
-0.1501, 0.6177, 0.4218;
0.7441, -0.0882, 0.1425;
0.6241, 0.2793, -0.1137;
0.8693, -0.0331, 0.0289;
-0.0154, -0.2706, 0.6262;
-0.0914, 0.0995, 0.7216;
0.1502, 0.0835, 0.398}.
I am trying to improve my loop computation speed by using foreach, but there is a simple Rcpp function I defined inside of this loop. I saved the Rcpp function as mproduct.cpp, and I call out the function simply using
sourceCpp("mproduct.cpp")
and the Rcpp function is a simple one, which is to perform matrix product in C++:
// [[Rcpp::depends(RcppArmadillo, RcppEigen)]]
#include <RcppArmadillo.h>
#include <RcppEigen.h>
// [[Rcpp::export]]
SEXP MP(const Eigen::Map<Eigen::MatrixXd> A, Eigen::Map<Eigen::MatrixXd> B){
Eigen::MatrixXd C = A * B;
return Rcpp::wrap(C);
}
So, the function in the Rcpp file is MP, referring to matrix product. I need to perform the following foreach loop (I have simplified the code for illustration):
foreach(j=1:n, .package='Rcpp',.noexport= c("mproduct.cpp"),.combine=rbind)%dopar%{
n=1000000
A<-matrix(rnorm(n,1000,1000))
B<-matrix(rnorm(n,1000,1000))
S<-MP(A,B)
return(S)
}
Since the size of matrix A and B are large, it is why I want to use foreach to alleviate the computational cost.
However, the above code does not work, since it provides me error message:
task 1 failed - "NULL value passed as symbol address"
The reason I added .noexport= c("mproduct.cpp") is to follow some suggestions from people who solved similar issues (Can't run Rcpp function in foreach - "NULL value passed as symbol address"). But somehow this does not solve my issue.
So I tried to install my Rcpp function as a library. I used the following code:
Rcpp.package.skeleton('mp',cpp_files = "<my working directory>")
but it returns me a warning message:
The following packages are referenced using Rcpp::depends attributes however are not listed in the Depends, Imports or LinkingTo fields of the package DESCRIPTION file: RcppArmadillo, RcppEigen
so when I tried to install my package using
install.packages("<my working directory>",repos = NULL,type='source')
I got the warning message:
Error in untar2(tarfile, files, list, exdir, restore_times) :
incomplete block on file
In R CMD INSTALL
Warning in install.packages :
installation of package ‘C:/Users/Lenovo/Documents/mproduct.cpp’ had non-zero exit status
So can someone help me out how to solve 1) using foreach with Rcpp function MP, or 2) install the Rcpp file as a package?
Thank you all very much.
The first step would be making sure that you are optimizing the right thing. For me, this would not be the case as this simple benchmark shows:
set.seed(42)
n <- 1000
A<-matrix(rnorm(n*n), n, n)
B<-matrix(rnorm(n*n), n, n)
MP <- Rcpp::cppFunction("SEXP MP(const Eigen::Map<Eigen::MatrixXd> A, Eigen::Map<Eigen::MatrixXd> B){
Eigen::MatrixXd C = A * B;
return Rcpp::wrap(C);
}", depends = "RcppEigen")
bench::mark(MP(A, B), A %*% B)[1:5]
#> # A tibble: 2 x 5
#> expression min median `itr/sec` mem_alloc
#> <bch:expr> <bch:tm> <bch:tm> <dbl> <bch:byt>
#> 1 MP(A, B) 277.8ms 278ms 3.60 7.63MB
#> 2 A %*% B 37.4ms 39ms 22.8 7.63MB
So for me the matrix product via %*% is several times faster than the one via RcppEigen. However, I am using Linux with OpenBLAS for matrix operations while you are on Windows, which often means reference BLAS for matrix operations. It might be that RcppEigen is faster on your system. I am not sure how difficult it is for Windows user to get a faster BLAS implementation (https://csgillespie.github.io/efficientR/set-up.html#blas-and-alternative-r-interpreters might contain some pointers), but I would suggest spending some time on investigating this.
Now if you come to the conclusion that you do need RcppEigen or RcppArmadillo in your code and want to put that code into a package, you can do the following. Instead of Rcpp::Rcpp.package.skeleton() use RcppEigen::RcppEigen.package.skeleton() or RcppArmadillo::RcppArmadillo.package.skeleton() to create a starting point for a package based on RcppEigen or RcppArmadillo, respectively.
I'm currently having a problem with dealing with linking between two modules with different baselines.
Example:
I have a module A with 10 baselines and module B with 10 baselines.
Links are out links from A => B. The last baseline of module A has been done 1 month after the last baseline of module B. In the meantime more objects in module B have been created and were linked from module A to them.
So now I have some links in the last baseline of module A that link to objects in module B that are non existent in the last baseline of module B (only are existent in the last baseline++).
When I run my DXL-Script in order to get all linked objects I also get the IDs of these non existent object eventhough they do not exist in the last baseline. I open both modules and load the last baseline with the load(module, baseline,false) function. The name of the baseline works and my debugging shows me that I'm really working on the latest baseline. But somehow these non existent objects are listed because of the linking. I've read something about echoed links but they do not work for me.
A possible solution would be to get the Date of the baseline of module A and compare it with the creation date of the linked object. But there must be an easier and cleaner way imho... any ideas?
I am sorry that I do not have a direct answer to your question. I tried a script between two modules (A and B) like you presented. I linked the objects, made a new baseline, and added a few more objects that was also linked up.
I do not meet the problem you presented when I run the following script. I hope there could be something that gives you a new idea. Basically I open a baseline of Module B and count in-links. Newer objects (which I find in current version) are not included in this count.
Module modB = read("/Testmappe/ModuleB", false)
Module baselineModuleB
Object objB
Link l
string linkModName = "*"
int countLinks = 0
// baseline(MAJ, MIN, SUFFIX)
Baseline myBaseline = baseline(0, 1, "")
if(baselineExists(modB, myBaseline))
{
baselineModuleB = load(modB, myBaseline, false)
for objB in baselineModuleB Module do {
for l in all(objB<-linkModName) do {
countLinks++
}
}
print "Number of in-links are: " countLinks ""
}
DISCLAIMER: This question is only for those who have access to the econometrics toolbox in Matlab.
The Situation: I would like to use Matlab to simulate N observations from an ARIMA(p, d, q) model using the econometrics toolbox. What's the difficulty? I would like the innovations to be simulated with deterministic, time-varying variance.
Question 1) Can I do this using the in-built matlab simulate function without altering it myself? As near as I can tell, this is not possible. From my reading of the docs, the innovations can either be specified to have a constant variance (ie same variance for each innovation), or be specified to be stochastically time-varying (eg a GARCH model), but they cannot be deterministically time-varying, where I, the user, choose their values (except in the trivial constant case).
Question 2) If the answer to question 1 is "No", then does anyone see any reason why I can't edit the simulate function from the econometrics toolbox as follows:
a) Alter the preamble such that the function won't throw an error if the Variance field in the input model is set to a numeric vector instead of a numeric scalar.
b) Alter line 310 of simulate from:
E(:,(maxPQ + 1:end)) = Z * sqrt(variance);
to
E(:,(maxPQ + 1:end)) = (ones(NumPath, 1) * sqrt(variance)) .* Z;
where NumPath is the number of paths to be simulated, and it can be assumed that I've included an error trap to ensure that the (input) deterministic variance path stored in variance is of the right length (ie equal to the number of observations to be simulated per path).
Any help would be most appreciated. Apologies if the question seems basic, I just haven't ever edited one of Mathwork's own functions before and didn't want to do something foolish.
UPDATE (2012-10-18): I'm confident that the code edit I've suggested above is valid, and I'm mostly confident that it won't break anything else. However it turns out that implementing the solution is not trivial due to file permissions. I'm currently talking with Mathworks about the best way to achieve my goal. I'll post the results here once I have them.
It's been a week and a half with no answer, so I think I'm probably okay to post my own answer at this point.
In response to my question 1), no, I have not found anyway to do this with the built-in matlab functions.
In response to my question 2), yes, what I have posted will work. However, it was a little more involved than I imagined due to matlab file permissions. Here is a step-by-step guide:
i) Somewhere in your matlab path, create the directory #arima_Custom.
ii) In the command window, type edit arima. Copy the text of this file into a new m file and save it in the directory #arima_Custom with the filename arima_Custom.m.
iii) Locate the econometrics toolbox on your machine. Once found, look for the directory #arima in the toolbox. This directory will probably be located (on a Linux machine) at something like $MATLAB_ROOT/toolbox/econ/econ/#arima (on my machine, $MATLAB_ROOT is at /usr/local/Matlab/R2012b). Copy the contents of #arima to #arima_Custom, except do NOT copy the file arima.m.
iv) Open arima_Custom for editing, ie edit arima_Custom. In this file change line 1 from:
classdef (Sealed) arima < internal.econ.LagIndexableTimeSeries
to
classdef (Sealed) arima_Custom < internal.econ.LagIndexableTimeSeries
Next, change line 406 from:
function OBJ = arima(varargin)
to
function OBJ = arima_Custom(varargin)
Now, change line 993 from:
if isa(OBJ.Variance, 'double') && (OBJ.Variance <= 0)
to
if isa(OBJ.Variance, 'double') && (sum(OBJ.Variance <= 0) > 0)
v) Open the simulate.m located in #arima_Custom for editing (we copied it there in step iii). It is probably best to open this file by navigating to it manually in the Current Folder window, to ensure the correct simulate.m is opened. In this file, alter line 310 from:
E(:,(maxPQ + 1:end)) = Z * sqrt(variance);
to
%Check that the input variance is of the right length (if it isn't scalar)
if isscalar(variance) == 0
if size(variance, 2) ~= 1
error('Deterministic variance must be a column vector');
end
if size(variance, 1) ~= numObs
error('Deterministic variance vector is incorrect length relative to number of observations');
end
else
variance = variance(ones(numObs, 1));
end
%Scale innovations using deterministic variance
E(:,(maxPQ + 1:end)) = sqrt(ones(numPaths, 1) * variance') .* Z;
And we're done!
You should now be able to simulate with deterministically time-varying variance using the arima_Custom class, for example (for an ARIMA(0,1,0)):
ARIMAModel = arima_Custom('D', 1, 'Variance', ScalarVariance, 'Constant', 0);
ARIMAModel.Variance = TimeVaryingVarianceVector;
[X, e, VarianceVector] = simulate(ARIMAModel, NumObs, 'numPaths', NumPaths);
Further, you should also still be able to use matlab's original arima class, since we didn't alter it.