I search the Caffe source code a bit, and the ReadImageToDatum function in caffe/src/caffe/util/io.cpp only supports int type labels.
I've also noticed that there's an array_to_datum function in caffe/python/caffe/io.py, which seems to not have the constraint on the type of label, but I'm not exactly sure how I'm supposed to use it.
How can I use numerical (non-integers) labels for regression?
I suppose Datum type was designed with image classification in mind.
For regression, I recommend using "HDF5Data" input layer.
See this answer for example.
Using hdf5 binary files allows for more flexibility in the number, shape and type of inputs you feed your net.
In addition to Shai's answer, caffe's Datum class only support int type labels.
In caffe/src/caffe/proto/caffe.proto it is defined
message Datum {
optional int32 channels = 1;
optional int32 height = 2;
optional int32 width = 3;
// the actual image data, in bytes
optional bytes data = 4;
optional int32 label = 5;
// Optionally, the datum could also hold float data.
repeated float float_data = 6;
// If true data contains an encoded image that need to be decoded
optional bool encoded = 7 [default = false];
}
So in the generated caffe.pb.h file it's like
private:
...
::google::protobuf::int32 channels_;
::google::protobuf::int32 height_;
::std::string* data_;
::google::protobuf::int32 width_;
::google::protobuf::int32 label_;
::google::protobuf::RepeatedField< float > float_data_;
bool encoded_;
...
Related
I'm trying to create a variable format specifier for use in $display/$write. I've tried a large number of things, but here is what I have at the moment.
What I want to end up with is: $display(format_var,data_1,data_2), where the format string is pre-calculated using $sformatf or other.
Code:
module test;
function void pprint(input int data_1,input int field_1,input int data_2,input int field_2);
string format;
begin
format = $sformatf("%0d'h%%%0dx,%0d'h%%%0dx",field_1,field_1/4,field_2,field_2/4);
$display("format = %s",format);
$display(format,data_1,data_2);
end
endfunction
initial
begin
pprint(5,8,73737229,128);
$stop;
end
endmodule
The output I expect is:
format = 8'h%2x,128'h%32x
8'h05,128'h000000000000000000000000465240D
The output I get is:
format = 8'h%2x,128'h%32x
8'h%2x,128'h%32x 5 73737229
What do I need to do? The simulator is Vivado 2020.3
Later:
Trying more things, the following function does do what I want. My conclusion is that $display/$write can't take a variable as the format string, but $sformatf can.
function void pprint(input int data_1,input int field_1,input int data_2,input int field_2);
string format;
string outstr;
begin
format = $sformatf("%0d'h%%%0dx,%0d'h%%%0dx",field_1,field_1/4,field_2,field_2/4);
$display("format = %s",format);
$display("%s",$sformatf(format,data_1,data_2));
end
endfunction
Try:
function void pprint(
input logic [4095:0] data_1,
input int field_1,
input logic [4095:0] data_2,
input int field_2 );
string format;
format = $sformatf("%0d'h%%%0dh,%0d'h%%%0dh",
field_1, (field_1+3)/4,
field_2, (field_2+3)/4 );
$display("format = %s",format);
$display($sformatf(format,data_1,data_2));
endfunction
This should give you the output:
format = 8'h%02h,128'h%032h
8'h05,128'h000000000000000000000000465240D
Adding a zero between the % and digit could tells the simulator to pad the upper bits with zeros.
For some reason $display(format,data_1,data_2) did not use the format on simulators on edaplayground, but it did work with $sformatf so I simply nested it.
I needed to increase the bit width of the input data otherwise it would show leading zeros over 8 digits. Adjust if necessary.
Adding 3 to the field is for handling non multiples of 4. It will always round down after division.
According to section 21.3.3 Formatting data to a string of the SystemVerilog LRM, only $sformat and $sformatf have a specific formatting argument that can be a string literal or string variable. All other output tasks like $display treat any string literal argument as format specifiers and do not interpret the strings inside string variables for formatting.
I'm getting a Bluetooth Characteristic from a Bluetooth controller with flutter blue as a List. This characteristic contains weight measurement of a Bluetooth scale. Is there a function to convert this list of ints to a Double?
I tried to find some background on float representations by reading the IEEE 754 standard. There is the dart library typed_data but I am new to dart and have no experience with this lib.
Example:
I have this List: [191, 100, 29, 173] which is coming from a bluetooth controller as a IEEE754 representation for a float value.
Now i believe i have to convert each int to hex and concat these values: bf 64 1d ad
Next thing need to do is convert this to double, but i cannot find a function to convert hex to double. Only int.parse("0xbf641dad").
I guess your mean to convert the list of ints to a list of floats, not to a single float, right?
First, dart has no type called float. Instead it has type double. To convert it, you can use the map() function:
var listInt = [1, 2, 3];
var listDouble = list.map((i) => i.toDouble()).toList();
Had this same issue.
I think you mean how do you make the four bytes into a float32 ?
I needed to do the same thing.
you will want to do something like this:
first take the List value as a byte buffer, then take the byte data, then you can use the getFloat32 function.
ByteBuffer buffer = new Int8List.fromList(value_in).buffer;
ByteData byteData = new ByteData.view(buffer);
result = byteData.getFloat32(0);
Just be a little aware that the order of bytes from the Bluetooth may be back to front, as the convention varys.
Also you will need the typed_data library:
import 'dart:typed_data'; //for data formatting
I'm trying to go in the other direction now . . . for the obvious reason.
You may need this:
double parseHexString(String hexStr, bool littleEndian){
if(hexStr.length % 2 != 0){
return 0;
}
if(littleEndian == true){
List<int> bytes = hex.decode(hexStr).reversed.toList();
hexStr = hex.encode(bytes);
}
var byteConvert = ByteData(12);
byteConvert.setInt64(0, int.parse(hexStr,radix: 16));
return byteConvert.getFloat64(0);}
And demo :
double lat = parseHexString("0000004069665E40",true);
//lat = 121.60017395019531
If you already have:
int.parse("0xbf641dad")
I don't see why this wouldn't work:
int.parse("0xbf641dad").toDouble();
I used dart:typed_data for this:
import 'dart:typed_data';
List<int> intList = [191, 100, 29, 173];
double asFloat = ByteData.view(Uint8List.fromList(List.from(intList)).buffer).getFloat32(0, Endian.little);
Library reference: https://api.dart.dev/be/136883/dart-typed_data/dart-typed_data-library.html
Is it save to use cast to int instead of Math.floor to convert float / double values to integers?
var scale = 1.5;
int foo1 = (int)scale;
int foo2 = Math.floor(scale);
In this case both Case to Int and Math.floor will return integer value. If x=3.5 then both function will return 3 in output. Cast to int is a function to convert variable of any datatype to integer type, on the other hand Math.floor function will only floor the decimal number to integer not converting datatype. But result will be different in case of negative values because Cast to Int approaches to zero and Math.floor approaches to negative infinity. So in that perspective if you are working on real numbers (both positive and negative) then it is unsafe to use Cast to Int instead of Math.floor to get precise output.
As Vala code is translated into C, this is the same question as Cast to int vs floor
TL;DR: yes it is safe but the result of Math.floor and float/double casting are different when negative numbers are given.
Note that Math.floor is part of the GLib library and thus is not available in the POSIX profile.
weight is a field (Number in Firestore), set as 100.
int weight = json['weight'];
double weight = json['weight'];
int weight works fine, returns 100 as expected, but double weight crashes (Object.noSuchMethod exception) rather than returning 100.0, which is what I expected.
However, the following works:
num weight = json['weight'];
num.toDouble();
When parsing 100 from Firestore (which actually does not support a "number type", but converts it), it will by standard be parsed to an int.
Dart does not automatically "smartly" cast those types. In fact, you cannot cast an int to a double, which is the problem you are facing. If it were possible, your code would just work fine.
Parsing
Instead, you can parse it yourself:
double weight = json['weight'].toDouble();
Casting
What also works, is parsing the JSON to a num and then assigning it to a double, which will cast num to double.
double weight = json['weight'] as num;
This seems a bit odd at first and in fact the Dart Analysis tool (which is e.g. built in into the Dart plugin for VS Code and IntelliJ) will mark it as an "unnecessary cast", which it is not.
double a = 100; // this will not compile
double b = 100 as num; // this will compile, but is still marked as an "unnecessary cast"
double b = 100 as num compiles because num is the super class of double and Dart casts super to sub types even without explicit casts.
An explicit cast would be the follwing:
double a = 100 as double; // does not compile because int is not the super class of double
double b = (100 as num) as double; // compiles, you can also omit the double cast
Here is a nice read about "Types and casting in Dart".
Explanation
What happened to you is the following:
double weight;
weight = 100; // cannot compile because 100 is considered an int
// is the same as
weight = 100 as double; // which cannot work as I explained above
// Dart adds those casts automatically
You can do it in one line:
double weight = (json['weight'] as num).toDouble();
You can Parse the data Like given below:
Here document is a Map<String,dynamic>
double opening = double.tryParse(document['opening'].toString());
In Dart, int and double are separate types, both subtypes of num.
There is no automatic conversion between number types. If you write:
num n = 100;
double d = n;
you will get a run-time error. Dart's static type system allows unsafe down-casts, so the unsafe assignment of n to d (unsafe because not all num values are double values) is treated implicitly as:
num n = 100;
double d = n as double;
The as double checks that the value is actually a double (or null), and throws if it isn't. If that check succeeds, then it can safely assign the value to d since it is known to match the variable's type.
That's what's happening here. The actual value of json['weight'] (likely with static type Object or dynamic) is the int object with value 100. Assigning that to int works. Assigning it to num works. Assigning it to double throws.
The Dart JSON parser parses numbers as integers if they have no decimal or exponent parts (0.0 is a double, 0e0 is a double, 0 is an integer). That's very convenient in most cases, but occasionally annoying in cases like yours where you want a double, but the code creating the JSON didn't write it as a double.
In cases like that, you just have to write .toDouble() on the values when you extract them. That's a no-op on actual doubles.
As a side note, Dart compiled to JavaScript represents all numbers as the JavaScript Number type, which means that all numbers are doubles. In JS compiled code, all integers can be assigned to double without conversion. That will not work when the code is run on a non-JS implementation, like Flutter, Dart VM/server or ahead-of-time compilation for iOS, so don't depend on it, or your code will not be portable.
Simply convert int to double like this
int a = 10;
double b = a + 0.0;
function encodeBmp(s:String){
s = Base64.Encode(s);
var width:Number = Math.ceil(Math.sqrt(s.length/4));
var bmp:BitmapData = new BitmapData(width,width,true,0x00000000);
var pos:Number=0; //track position in string
for(var x:Number=0;x<width;x++){
for(var y:Number=0;y<width;y++){
var col="0x";
for(var i:Number=0;i<4;i++){
col+=getHex(s.charAt(pos));
pos++;
}
bmp.setPixel32(x,y,col);
trace(col + " > 0x" + bmp.getPixel32(x,y).toString(16));
}
}
return bmp.clone();
}
Basically, the trace statement returns this:
0x56326868 > 0x56326868
0x64434270 > 0x64424270
0x63794230 > 0x63794331
...
Why is the result of getPixel32 different from that of the set value, and how can I resolve this?
EDIT: getPixel is accurate, but doesn't have the extra bit that get/setPixel32 does... I would prefer to have more data per pixel.
You're Base64 encoding a String to store it in a BitmapData. I hope you didn't know that, or are ashamed.
Anyway, here is what I see happening:
First, if you type col as a String, you'll get a compile error on setPixel32, because it's expecting a uint.
Now, we can ignore all the looping and hex strings to get some simple code to reproduce the problem.
var bitmapData:BitmapData = new BitmapData(1, 1);
bitmapData.setPixel32(0, 0, 1682129520);
trace(bitmapData.getPixel32(0, 0));
By using an int literal, we can see that it's not a problem with type coercion, but something inside of BitmapData. This makes sense, since BitmapData is intended to store graphics data, so storing the graphics in a way that is graphically similar but not exact is fine.
You're probably seeing the side-effects of pre-multiplied alpha in Flash. This would also explain why you don't see any problems with getPixel and setPixel.
You can read more about it here: http://www.quasimondo.com/archives/000665.php
So how do you fix the problem? Use the correct data type. A ByteArray is designed to store binary data.