I am using FANN for function approximation. My code is here:
/*
* File: main.cpp
* Author: johannsebastian
*
* Created on November 26, 2013, 8:50 PM
*/
#include "../FANN-2.2.0-Source/src/include/doublefann.h"
#include "../FANN-2.2.0-Source/src/include/fann_cpp.h"
//#include <doublefann>
//#include <fann/fann_cpp>
#include <cstdlib>
#include <iostream>
using namespace std;
using namespace FANN;
//Remember: fann_type is double!
int main(int argc, char** argv) {
//create a test network: [1,2,1] MLP
neural_net * net = new neural_net;
const unsigned int layers[3] = {1, 2, 1};
net->create_standard_array(3, layers);
//net->create_standard(num_layers, num_input, num_hidden, num_output);
//net->set_learning_rate(0.7f);
//net->set_activation_steepness_hidden(0.7);
//net->set_activation_steepness_output(0.7);
net->set_activation_function_hidden(SIGMOID);
net->set_activation_function_output(SIGMOID);
net->set_training_algorithm(TRAIN_RPROP);
//cout<<net->get_train_error_function()
//exit(0);
//test the number 2
fann_type * testinput = new fann_type;
*testinput = 2;
fann_type * testoutput = new fann_type;
*testoutput = *(net->run(testinput));
double outputasdouble = (double) *testoutput;
cout << "Test output: " << outputasdouble << endl;
//make a training set of x->x^2
training_data * squaredata = new training_data;
squaredata->read_train_from_file("trainingdata.txt");
//cout<<testinput[0]<<endl;
//cout<<testoutput[0]<<endl;
cout<<*(squaredata->get_input())[9]<<endl;
cout<<*(squaredata->get_output())[9]<<endl;
cout<<squaredata->length_train_data();
//scale data
fann_type * scaledinput = new fann_type[squaredata->length_train_data()];
fann_type * scaledoutput = new fann_type[squaredata->length_train_data()];
for (unsigned int i = 0; i < squaredata->length_train_data(); i++) {
scaledinput[i] = *squaredata->get_input()[i]/200;///100;
scaledoutput[i] = *squaredata->get_output()[i]/200;///100;
cout<<"In:\t"<<scaledinput[i]<<"\t Out:\t"<<scaledoutput[i]<<endl;
}
net->train_on_data(*squaredata, 1000000, 100000, 0.001);
*testoutput = *(net->run(testinput));
outputasdouble = (double) *testoutput;
cout << "Test output: " << outputasdouble << endl;
cout << endl << "Easy!";
return 0;
}
Here's trainingdata.txt:
10 1 1
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
10 100
When I run I get this:
Test output: 0.491454
10
100
10In: 0.005 Out: 0.005
In: 0.01 Out: 0.02
In: 0.015 Out: 0.045
In: 0.02 Out: 0.08
In: 0.025 Out: 0.125
In: 0.03 Out: 0.18
In: 0.035 Out: 0.245
In: 0.04 Out: 0.32
In: 0.045 Out: 0.405
In: 0.05 Out: 0.5
Max epochs 1000000. Desired error: 0.0010000000.
Epochs 1. Current error: 2493.7961425781. Bit fail 10.
Epochs 100000. Current error: 2457.3000488281. Bit fail 9.
Epochs 200000. Current error: 2457.3000488281. Bit fail 9.
Epochs 300000. Current error: 2457.3000488281. Bit fail 9.
Epochs 400000. Current error: 2457.3000488281. Bit fail 9.
Epochs 500000. Current error: 2457.3000488281. Bit fail 9.
Epochs 600000. Current error: 2457.3000488281. Bit fail 9.
Epochs 700000. Current error: 2457.3000488281. Bit fail 9.
Epochs 800000. Current error: 2457.3000488281. Bit fail 9.
Epochs 900000. Current error: 2457.3000488281. Bit fail 9.
Epochs 1000000. Current error: 2457.3000488281. Bit fail 9.
Test output: 1
Easy!
RUN FINISHED; exit value 0; real time: 9s; user: 10ms; system: 4s
Why is the training not working? After I asked a similar question, I was told to scale the NN's input and output. I have done so. Am I getting some parameter(s) wrong, or do I simply have to train longer?
The node number in your hidden layer is too few to fit a quadratic function. I would try 10.
Besides, I would like to recommend you a fun applet in which you can simulate the training process by parameter setting. I tried with 10 hidden layer nodes and unipolar sigmoid as both hidden layer and output layer activation function, the fitting is not bad (but randomize the weights may lead to the failure of converge, so more nodes in hidden layer are highly recommended, you can try to play this applet yourself and observe some interesting points):
Maybe a bit late, but maybe new FANN beginner will see this answer, I hope this helps !
I think your problem comes from the data format in your trainingdata.txt:
See :
FANN data format
You have to do a newline after each input and each output.
In your case, you have 10 examples with 1 input and 1 output. Then, you have to format your file like this :
10 1 1
1
1
2
4
3
9
4
16
5
25
6
36
...
Note : I notice when the data format is wrong, the error computed by training method is very (very) high. Could be an hint to look at your file format when you see huge error value.
Related
I am doing machine learning.Here I want to find the best triple (max_samples, n_trees and threshold) that gives the greatest performance in terms of area under ROC curve and area under recall precison curve.
Here is the code:
def meilleur_triplet(x,classes):
for n_trees in np.arange(100,160,10):
for sample_size in np.arange(0.1,1,0.1):
for threshold in np.arange(0.4,1,0.1):
model=IforestLocal(sample_size,n_trees)
model.fit(x)
y_pred,y_score=model.predict(x,threshold)
auc=roc_auc_score(classes,y_pred)
auc_pr=average_precision_score(classes,y_pred)
Now when I use max_samples with a range of int I don't have an error however if it's in float I have the following error:
**
TypeError Traceback (most recent call last)
Input In [201], in <cell line: 1>()
----> 1 meilleur_triplet(X_glass,y_glass)
Input In [200], in meilleur_triplet(x, classes)
6 for threshold in np.arange(0.4,1,0.1):#(0.4,1,0.1)
8 model=IforestLocal(sample_size,n_trees)
----> 9 model.fit(x)
File ~\Desktop\THESE\Maurras\Code_Maurras\iforest_D.py:45, in IsolationForest.fit(self, X)
42 self.sample_size = len_x
44 for i in range(self.n_trees):
---> 45 sample_idx = random.sample(list(range(len_x)), self.sample_size)
46 # TODO: Must be deleted before compute the memory consumption of the methods
47 self.samples.append(sample_idx)
File ~\anaconda3\lib\random.py:450, in Random.sample(self, population, k, counts)
448 if not 0 <= k <= n:
449 raise ValueError("Sample larger than population or is negative")
--> 450 result = [None] * k
451 setsize = 21 # size of a small set minus size of an empty list
452 if k > 5:
TypeError: can't multiply sequence by non-int of type 'numpy.float64'
**
This is where I called the function
meilleur_triplet(X_glass,y_glass)
Thank you please help me
I'm working with an Octave script to process data files with high sample rates (up to 200kS/s collected over 3 minutes). The code runs into issues when processing any files with a sample rate above 50kS/s, regardless of size or number of samples but functions correctly otherwise.
The error I receive when attempting to run the code with files above 50kS/s is called from the hist function:
error: x(0): subscripts must be either integers 1 to (2^63)-1 or logicals
error:
I have narrowed the cause to the following section of code (note that FS is the detected sampling frequency):
FILTER_ORDER = 1;
FILTER_CUTOFF = 1 / (2 * pi * 300e-3);
[b_lp, a_lp] = butter(FILTER_ORDER, FILTER_CUTOFF / (FS / 2), 'low');
%
s = SCALING_FACTOR * filter(b_lp, a_lp, u_q) ;
P = s ;
%
tau = 20;
transient = tau * FS; % index after the transient
Pmax = max(P(transient:end));
%
s = s(transient:end);
%
NUMOF_CLASSES = 10000; % number of bins used for the histogram
[bin_cnt, cpf.magnitude] = hist(s, NUMOF_CLASSES); % sorts data into the number of bins specified
I can try to provide more information if required, I'm not very familiar with Octave.
I'm getting an error while trying to multiply a vector component with an array (element-wise multiplication or broadcast). The docs show that this overloaded case for * should be fine:
AFAPI array operator* (const float &lhs, const array &rhs)
Multiplies two arrays or an array and a value. (const array&, const
array&)
But according to the error message below, perhaps vect(0) needs to be further flattened or reduced so that the sizes are consistent?
The error statement is clear:
Invalid dimension for argument 1 Expected: ldims == rides
Below is the code:
#include <arrayfire.h>
int main(int argc, char *argv[])
{
int device = argc > 1 ? atoi(argv[1]) : 0;
af::setDevice(device);
af::info();
int n = 3;
int N = 5;
// Create the arrays:
af::array matrix = af::constant(0,n,n,f32); // 3 x 3 float array of zeros
af::array vect = af::seq(1,N); // A col vector of floats: {1.0, ... ,5.0}
// Show the arrays:
af_print(matrix);
af_print(vect);
// Print a single component of the vector:
af_print(vect(0));
// This line produces the error (see below):
af_print(vect(0) * matrix); // Why doesn't this work?
// But somthing like this is fine:
af_print(1.0 * matrix);
return 0;
}
Producing the output:
ArrayFire v3.3.2
ATI Radeon HD 6750M
matrix [3 3 1 1]
0.0000 0.0000 0.0000
0.0000 0.0000 0.0000
0.0000 0.0000 0.0000
vect [5 1 1 1]
1.0000
2.0000
3.0000
4.0000
5.0000
vect(0) [1 1 1 1]
1.0000
The dims() output of af_print() for the matrix = [3 3 1 1], and vect(0) = [1 1 1 1], make me suspicious, but I'm not sure how to flatten further. One would think this example to be a common way of using the ArrayFire API.
The error exception that is thrown is:
libc++abi.dylib: terminating with uncaught exception of type
af::exception: ArrayFire Exception (Invalid input size:203): In
function getOutDims In file src/backend/ArrayInfo.cpp:173
Invalid dimension for argument 1 Expected: ldims == rides
In function af::array af::operator*(const af::array &, const af::array
&)
Adding a use-case to clarify:
In practice I am constructing a final array by summation of coeff(k) * (a 2-d slice of a 3-d array Z):
for (int j = 0; j<indx.dims(0); ++j)
final += coeff(indx(j)) * Z(af::span,af::span,indx(j));
I'll look into using a gfor but initially just wanted to get the correct numerical output. Note also that the vector: index is predefined, e.g., say index = {1, 2, 4, 7, ...} and the elements are not necessarily in sequence; this allows the selection of specific terms.
ArrayFire does not implicitly do vector array-scalar array element-wise operation (the case you say is failing). Only vector array-value ones are supported implicitly.
To do what you are doing, you will need to use the tile() function as shown below.
af_print(tile(vect(0), matrix.dims()) * matrix);
Since the dimension being tiled is 1, tile will be used as a JIT function. There is no extra memory used here. The entire computation is done in a single kernel. Hence no performance hit either.
Since OP added a usecase since the last answer, this is how you write a fully vectorized version in arrayfire.
array coeffs = moddims(coeff(indx), 1, 1, coeff.elements());
array final = sum(Z(span, span, indx) * tile(coeffs, Z.dims(0), Z.dims(1)), 2);
I am trying to implement logistic regression with gradient descent,
I get my Cost function j_theta for the number of iterations and fortunately my j_theta is decreasing when plotted j_theta against the number of iteration.
The data set I use is given below:
x=
1 20 30
1 40 60
1 70 30
1 50 50
1 50 40
1 60 40
1 30 40
1 40 50
1 10 20
1 30 40
1 70 70
y= 0
1
1
1
0
1
0
0
0
0
1
The code that I managed to write for logistic regression using Gradient descent is:
%1. The below code would load the data present in your desktop to the octave memory
x=load('stud_marks.dat');
%y=load('ex4y.dat');
y=x(:,3);
x=x(:,1:2);
%2. Now we want to add a column x0 with all the rows as value 1 into the matrix.
%First take the length
[m,n]=size(x);
x=[ones(m,1),x];
X=x;
% Now we limit the x1 and x2 we need to leave or skip the first column x0 because they should stay as 1.
mn = mean(x);
sd = std(x);
x(:,2) = (x(:,2) - mn(2))./ sd(2);
x(:,3) = (x(:,3) - mn(3))./ sd(3);
% We will not use vectorized technique, Because its hard to debug, We shall try using many for loops rather
max_iter=50;
theta = zeros(size(x(1,:)))';
j_theta=zeros(max_iter,1);
for num_iter=1:max_iter
% We calculate the cost Function
j_cost_each=0;
alpha=1;
theta
for i=1:m
z=0;
for j=1:n+1
% theta(j)
z=z+(theta(j)*x(i,j));
z
end
h= 1.0 ./(1.0 + exp(-z));
j_cost_each=j_cost_each + ( (-y(i) * log(h)) - ((1-y(i)) * log(1-h)) );
% j_cost_each
end
j_theta(num_iter)=(1/m) * j_cost_each;
for j=1:n+1
grad(j) = 0;
for i=1:m
z=(x(i,:)*theta);
z
h=1.0 ./ (1.0 + exp(-z));
h
grad(j) += (h-y(i)) * x(i,j);
end
grad(j)=grad(j)/m;
grad(j)
theta(j)=theta(j)- alpha * grad(j);
end
end
figure
plot(0:1999, j_theta(1:2000), 'b', 'LineWidth', 2)
hold off
figure
%3. In this step we will plot the graph for the given input data set just to see how is the distribution of the two class.
pos = find(y == 1); % This will take the postion or array number from y for all the class that has value 1
neg = find(y == 0); % Similarly this will take the position or array number from y for all class that has value 0
% Now we plot the graph column x1 Vs x2 for y=1 and y=0
plot(x(pos, 2), x(pos,3), '+');
hold on
plot(x(neg, 2), x(neg, 3), 'o');
xlabel('x1 marks in subject 1')
ylabel('y1 marks in subject 2')
legend('pass', 'Failed')
plot_x = [min(x(:,2))-2, max(x(:,2))+2]; % This min and max decides the length of the decision graph.
% Calculate the decision boundary line
plot_y = (-1./theta(3)).*(theta(2).*plot_x +theta(1));
plot(plot_x, plot_y)
hold off
%%%%%%% The only difference is In the last plot I used X where as now I use x whose attributes or features are featured scaled %%%%%%%%%%%
If you view the graph of x1 vs x2 the graph would look like,
After I run my code I create a decision boundary. The shape of the decision line seems to be okay but it is a bit displaced. The graph of the x1 vs x2 with decision boundary is given below:
![enter image description here][2]
Please suggest me where am I going wrong ....
Thanks:)
The New Graph::::
![enter image description here][1]
If you see the new graph the coordinated of x axis have changed ..... Thats because I use x(feature scalled) instead of X.
The problem lies in your cost function calculation and/or gradient calculation, your plotting function is fine. I ran your dataset on the algorithm I implemented for logistic regression but using the vectorized technique because in my opinion it is easier to debug.
The final values I got for theta were
theta =
[-76.4242,
0.8214,
0.7948]
I also used alpha = 0.3
I plotted the decision boundary and it looks fine, I would recommend using the vectorized form as it is easier to implement and to debug in my opinion.
I also think your implementation of gradient descent is not quite correct. 50 iterations is just not enough and the cost at the last iteration is not good enough. Maybe you should try to run it for more iterations with a stopping condition.
Also check this lecture for optimization techniques.
https://class.coursera.org/ml-006/lecture/37
I'm trying the second day to send a midi signal. I'm using following code:
int pitchValue = 8191 //or -8192;
int msb = ?;
int lsb = ?;
UInt8 midiData[] = { 0xe0, msb, lsb};
[midi sendBytes:midiData size:sizeof(midiData)];
I don't understand how to calculate msb and lsb. I tried pitchValue << 8. But it's working incorrect, When I'm looking to events using midi tool I see min -8192 and +8064 max. I want to get -8192 and +8191.
Sorry if question is simple.
Pitch bend data is offset to avoid any sign bit concerns. The maximum negative deviation is sent as a value of zero, not -8192, so you have to compensate for that, something like this Python code:
def EncodePitchBend(value):
''' return a 2-tuple containing (msb, lsb) '''
if (value < -8192) or (value > 8191):
raise ValueError
value += 8192
return (((value >> 7) & 0x7F), (value & 0x7f))
Since MIDI data bytes are limited to 7 bits, you need to split pitchValue into two 7-bit values:
int msb = (pitchValue + 8192) >> 7 & 0x7F;
int lsb = (pitchValue + 8192) & 0x7F;
Edit: as #bgporter pointed out, pitch wheel values are offset by 8192 so that "zero" (i.e. the center position) is at 8192 (0x2000) so I edited my answer to offset pitchValue by 8192.