Related
I am trying to implement a weighted sampler for a very imbalanced data set. There are 182 different classes. Here is an array of the bin counts per class:
array([69487, 5770, 5753, 138, 4308, 10, 1161, 29, 5611,
350, 7, 183, 218, 4, 3, 3872, 5, 950,
33, 3, 443, 16, 20, 330, 4353, 186, 19,
122, 546, 6, 44, 6, 3561, 2186, 3, 48,
8440, 338, 9, 610, 74, 236, 160, 449, 72,
6, 37, 1729, 2255, 1392, 12, 1, 3426, 513,
44, 3, 28, 12, 9, 27, 5, 75, 15,
3, 21, 549, 7, 25, 871, 240, 128, 28,
253, 62, 55, 12, 8, 57, 16, 99, 6,
5, 150, 7, 110, 8, 2, 1296, 70, 1927,
470, 1, 1, 511, 2, 620, 946, 36, 19,
21, 39, 6, 101, 15, 7, 1, 90, 29,
40, 14, 1, 4, 330, 1099, 1248, 1146, 7414,
934, 156, 80, 755, 3, 6, 6, 9, 21,
70, 219, 3, 3, 15, 15, 12, 69, 21,
15, 3, 101, 9, 9, 11, 6, 32, 6,
32, 4422, 16282, 12408, 2959, 3352, 146, 1329, 1300,
3795, 90, 1109, 120, 48, 23, 9, 1, 6,
2, 1, 11, 5, 27, 3, 7, 1, 3,
70, 1598, 254, 90, 20, 120, 380, 230, 180,
10, 10])
In some classes, instances are as low as 1. I am trying to implement a Weighted random sampler from torch for this dataset. However, as the class imbalance is so large, when I calculate weights using
count_occr = np.bincount(dataset.y)
lbl_weights = 1. / count_occr
weights = np.array(lbl_weights)
weights = torch.from_numpy(weights)
sampler = WeightedRandomSampler(weights.type('torch.DoubleTensor'), len(weights*2))
I get two error messages:
RuntimeWarning: divide by zero encountered in true_divide
and
RuntimeError: invalid multinomial distribution (encountering probability entry = infinity or NaN)
Does anyone have a work around for this ? I was considering multiplying the lbl_weights by some scalar however I am not sure if this is a viable option.
I have this simple code that doesn't compile.
const s = [_][_]int {
[_]int{08, 02, 22, 97, 38, 15, 00},
[_]int{49, 49, 99, 40, 17, 81, 18},
[_]int{81, 49, 31, 73, 55, 79, 14},
[_]int{52, 70, 95, 23, 04, 60, 11},
[_]int{22, 31, 16, 71, 51, 67, 63},
[_]int{24, 47, 32, 60, 99, 03, 45},
[_]int{32, 98, 81, 28, 64, 23, 67},
[_]int{67, 26, 20, 68, 02, 62, 12},
[_]int{24, 55, 58, 05, 66, 73, 99},
[_]int{21, 36, 23, 09, 75, 00, 76}
};
pub fn main() void
{
const w = s[0].len;
const h = s.len;
}
The compiler says:
./a.zig:1:14: error: inferred array size invalid here
const s = [_][_]int {
^
./a.zig:16:15: note: referenced here
const w = s[0].len;
What is the problem?
I'd be interested to know there's a deeper reason, but my simple understanding is that the current syntax [N]T allows for the array size to be elided using _, but not for more than one dimension.
So you can fix your problem using the following (N.B. I've used u8 because I'm unsure what your int is):
const s = [_][7]u8{
// Your items
}
I suspect this is because of the way the parsing rules are applied, so [7]u8 would be the type your nested array would hold, and will be used by the compiler to check contents are all of type [7]u8; you can confirm this by modifying one of your rows to have 6 elements and examining the resulting error.
If you want a variable number of items, you could start to look into an array of slices: [_][]u8, but I don't think that's what you're currently after.
This question already has answers here:
Create array of n items based on integer value
(6 answers)
Closed 4 years ago.
Need to create an array of 1 to n numbers with a single line of code in ruby.
I have tried it using while loop. But I'm sure there are other simpler way of doing this in ruby.
a = []
b = 1
while b < 100 do
a << b
b += 1
end
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
Convert a range into an array.
(1..n).to_a
another way
You can just splat a range:
[*1..n]
example
[*1..10]
=>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Or
a= Array(0..10)
puts a # => =>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
connecting TCP Socket server and sending Request. and also Server sends the response in Byte array. How to read byte array data in dart.
Socket.connect('localhost', 8081)
.then((socket) {
//Establish the onData, and onDone callbacks
socket.listen((data) {
print(new String.fromCharCodes(data).trim()); //Here data is byte[]
//How to read byte array data
},
onDone: () {
print("Done");
// socket.destroy();
},
onError: (e) {
print('Server error: $e');
});
socket.add([255, 12, 0, 11, 0, 9, 34, 82, 69, 70, 84, 65, 72, 73, 76]);
});
}
It depends on with data type was encoded to bytes. Let's suppose it's String
Then you can do it with dart:convert library.
import 'dart:convert' show utf8;
final decoded = utf8.decode(data);
It's pretty clear that there's a message structure in those bytes. You give two examples of messages:
[255, 12, 0, 11, 0, 9, 34, 82, 69, 70, 84, 65, 72, 73, 76]
and
[255, 20, 0, 11, 0, 0, 0, 15, 80, 82, 69, 77, 84, 65, 72, 73, 76, 45, 53, 53, 57, 55, 48]
Both start with 255, followed by what looks like two or three little endian 16 bit words (12 and 11) and (20, 11 and 0) followed by a string, who's length is encoded in a leading byte. If you are expected to inter-operate with another system, you really need the protocol spec.
Assuming I've guessed the structure correctly, this code
main() {
Uint8List input = Uint8List.fromList([
255,
20,
0,
11,
0,
0,
0,
15,
80,
82,
69,
77,
84,
65,
72,
73,
76,
45,
53,
53,
57,
55,
48
]);
ByteData bd = input.buffer.asByteData();
print(bd.getUint16(1, Endian.little)); // print the first short
print(bd.getUint16(3, Endian.little)); // and the second
print(bd.getUint16(5, Endian.little)); // and the third
int stringLength = input[7]; // get the length of the string
print(utf8.decode(input.sublist(8, 8 + stringLength))); // decode the string
}
produces
20
11
0
PREMTAHIL-55970
as expected
How to convert signed array of [Int8] to unsigned array of [UInt8].
let arryData: [Int8] = [-108, 11, -107, -14, 35, -57, -116, 118, 54, 91, 12, 67, 21, 29, -44, 111]
I just want to convert this above into array of Unsigned [UInt8]. How to achieve this in swift.? Thanks in advance.
If your intention is to convert signed 8-bit integers to
unsigned ones with the same bit representation (e.g. -1 -> 255):
let intArray: [Int8] = [0, 1, 2, 127, -1, -2, -128]
let uintArray = intArray.map { UInt8(bitPattern: $0) }
print(uintArray)
// [0, 1, 2, 127, 255, 254, 128]
[Int8] -> [UInt8]
You haven't specified how you want to treat negative values; by flipping them to their positive counterpart or by removing them. Below follows both cases.
Transforming negative values to positive ones by flipping sign:
let arrayData: [Int8] = [-108, 11, -107, -14, 35, -57, -116, 118, 54, 91, 12, 67, 21, 29, -44, 111]
let arrayDataUnsigned = arrayData.map { UInt8(abs($0)) }
/* [108, 11, 107, 14, 35, 57, 116, 118, 54, 91,
12, 67, 21, 29, 44, 111] */
Or, by removing the negative values:
let arrayDataUnsigned = arrayData.flatMap { $0 < 0 ? nil : UInt8($0) }
/* [11, 35, 118, 54, 91, 12, 67, 21, 29, 111] */