I am doing a project that involves generating 1 billion random 0/1 numbers and storing them in a file. This is my code:
import 'dart:io';
import 'dart:math';
final int COUNT = 1000000000;
final Random randomizer = new Random();
final File file = new File('./dart/dart.txt');
void main() {
if (file.existsSync()) {
file.deleteSync();
}
file.createSync();
DateTime before = new DateTime.now();
IOSink sink = file.openWrite(mode: FileMode.append);
for (int i = 0; i < COUNT; i++) {
sink.write(nextNumber().toString());
}
sink.close();
DateTime after = new DateTime.now();
print(after.millisecondsSinceEpoch - before.millisecondsSinceEpoch);
}
int nextNumber() {
return randomizer.nextInt(2);
}
It works fine with COUNT=10000, taking a dozen milliseconds to generate 10000 random numbers. However, as I raise COUNT to 1 billion, it took about 25GB of my RAM and ran for more than 40 minutes non-stop. I had to kill the process instead of letting it continue running.
How can I fix this?
P.S. I can't change the programming language since using different programming languages to generate such numbers are one aspect of my project. I also use C++, Java, Javascript, PS, Python, etc. to generate such big data, and none of them met this problem. The final file storing the data is less than 1 GB.
Related
I am trying to add an indicator code. While everything compiles just fine, the terminal shows the error, "array out of range".
This is the code that reads the price data from the .csv file and draws it as an indicator on the current chart.
The .csv file contains three months' price data in the form of 1 minute period.
#property indicator_buffers 1
#property indicator_color1 White
double ExtMapBuffer1[];
bool Calculated=false;
extern datetime endDateTime=D'2021.03.31 05:59:00';
int init()
{
SetIndexStyle(0,DRAW_LINE,0,1);
SetIndexBuffer(0,ExtMapBuffer1);
return(0);
}
int start()
{
int i,counted_bars=IndicatorCounted();
if (!Calculated)
{
double PriceArray[24*60*30*1]; **//issue1**
string lastDT = endDateTime;
if (!ReadCSVToArray(PriceArray,lastDT,fileName,"","\n")) Print("Reading file error: ",GetLastError());
int bars=ArraySize(PriceArray);
int firstBar = iBarShift(symbolName,PERIOD_M1,endDateTime);
ExtMapBuffer1[bars+firstBar-1]=PriceArray[0]; **//error1: array out of range**
for (i=2;i<=bars;i++)
{
ExtMapBuffer1[bars+firstBar-i]=PriceArray[i-1];
}
for (i=firstBar-1;i>=0;i--)
{
ExtMapBuffer1[i]=0;
}
Calculated=true;
}
return(0);
}
issue1: If I set the size of PriceArray as a big value for 3 month's data, such as 246030*3, I could meet the compile error: "local variables is larger than 512KB"
error1: If the size of PriceArray is small, the compiling is fine but running error.
Any help is appreciated.
To correctly read a csv's of variable or unknown length, you should use the first dimension of the array for your rows and the second dimension for your columns. You would therefore have an array statement of string data[1][5]; for a csv which contained date/O/H/L/C for example.
You can then resize your array as you loop through reading your csv file as follows:
int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");
if(handle==INVALID_HANDLE) Print("Error opening file ",GetLastError());
else
{
int row=0;
while(!FileIsEnding(handle))
{
if(row>0) ArrayResize(data,row+1);
//carry out array reading here
data[row][0]=FileReadString(handle); //date
data[row][1]=FileReadString(handle); //open
data[row][2]=FileReadString(handle); //high
data[row][3]=FileReadString(handle); //low
data[row][4]=FileReadString(handle); //close
row++;
}
FileClose(handle);
}
Note, you can not resize the 2nd dimension of a multi dimensional array so this dimension must match the structure of your csv file.
I was wondering if I could make an app in which time is display in words instead of conventional numbers. Do you think that there is any package for dart to convert time or numbers to words?
For example
1:30 AM will be One : Thirty AM
Thanks for reading this question. Have a wonderful day.
You can use any of available package to convert from number to words example
package: https://pub.dev/packages/number_to_words
import 'package:number_to_words/number_to_words.dart';
main(){
String input = '1:30 AM';
var s1 = input.split(':');
var s2 = s1[1].split(' ');
String hour = s1[0];
String minute = s2[0];
String hourWord = NumberToWord().convert('en-in',int.parse(hour));
String minuteWord = NumberToWord().convert('en-in',int.parse(minute));
print('$hourWord:$minuteWork ${s2[1]}');
}
How do you write a Binary Literal in Dart?
I can write a Hex Literal like so:
Int Number = 0xc
If I try the conventional way to write a Binary Literal:
Int Number = 0b1100
I get an error. I've tried to look it up, but I've not been able to find any information other than for hex.
There are currently no built-in binary number literals in Dart (or any base other than 10 and 16).
The closest you can get is: var number = int.parse("1100", radix: 2);.
Maybe you can use this:
// 0b1100 -> 1 at 3th bit and 1 at 2nd bit
final number = 1 << 3 | 1 << 2;
// Print binary string
print(number.toRadixString(2)) // 1100
Try binary package:
import 'package:binary/binary.dart';
void main() {
// New API.S
print(0x0C.toBinaryPadded(8)); // 00001100
}
see: https://pub.dev/documentation/binary/latest/
Given a Map, assignment, what is the fastest way to check if it contains any duplicate values in Dart? I am currently using a Set formed from the Map's values and checking its length against the original Map, which works of course, but I'm wondering if there's an especially performant alternative.
Set d = new Set.from(assignment.values);
if (d.length < assignment.length) {
return false; // indicates has duplicates in this context
}
EDIT:
Tried #mezoni's solution modified for my program, but it actually ran a bit slower than my original version. It probably has more to do with constant times than anything else.
List values = new List.from(assignment.values);
Set set = new Set();
for (var i = 0; i < assignment.length; i++) {
if (!set.add(values[i])) {
return false;
}
}
Complexity wise you won't be able to get anything faster. Creating the Set and filling it with the values of the Map is linear in the number of elements. Clearly you have to run through all the values, so you can't do any better than that.
Maybe you could find a solution with a smaller constant factor, but that's not clear. In particular for larger sets I think the Set solution is pretty efficient.
This is more of a algorithms question than a Dart question. In any case, you have to check every value against the others, giving n-1 + n-2 + ... + n-(n-1) checks, or n^2/2. Programmatically, it's easy to create a set, but you could also generate an array, sort the array, and then iterate once to check for duplicates. That finishes in O(n log n).
Fastets way (if you realy need better performance):
void main() {
// Values from map
var values = [1,2,3,2,1,3,2,1];
var length = values.length;
var set = new Set();
var duplicate = false;
// Only for statistics purpose
var statistics = 0;
for(var i = 0; i < length; i++) {
statistics++;
if(!set.add(values[i])) {
duplicate = true;
break;
}
}
print("Duplicate: $duplicate");
print("Performed in ${statistics} iteration(s) from $length possible");
}
Output:
Duplicate: true
Performed in 4 iteration(s) from 8 possible
P.S.
The first example can be recommended to use with List values.
But because Map.values not a List but Iterable then it would be more efficient do not convert them to List but use as is.
Here is modified sample for use with Iterable objects.
It will be work faster because in this algorithm not required convert all values to the List object because it not want using of all elements without exception.
Instead it wants use as less as possible access operation on original source. If the source supports lazy operation of the access to values (as Iterable) this will be even better.
void main() {
// Values from map
var values = [1,2,3,2,1,3,2,1];
var assignment = {};
var length = values.length;
var key = 0;
for(var value in values) {
assignment[key++] = value;
}
var set = new Set();
var duplicate = false;
// Only for statistics purpose
var statistics = 0;
for(var value in assignment.values) {
statistics++;
if(!set.add(value)) {
duplicate = true;
break;
}
}
print("Duplicate: $duplicate");
print("Performed in ${statistics} iteration(s) from $length possible");
}
How to get the length of a string array like
str 30 name[];//dynamic array
I used the following for getting the length,but it showing the error as "the variable is not of the type CLASS."
int len=name.get_length();
It sounds like you might be happier using the Array collection class.
http://msdn.microsoft.com/en-us/library/array.aspx
static void TestArray(Args _args)
{
Array strArray = new Array(Types::String);
;
strArray.value(1, 'abc');
strArray.value(2, 'def');
info(strfmt("%1", strArray.lastIndex()));
}
You need the dimOf function. Take a look to the reference:
http://msdn.microsoft.com/en-us/library/aa597117.aspx
Sorry, there is no build-in function to return the string array size. Since you are in full control what you put in the array, there need not be any!
The built-in function dimof returns the allocated size of the array, which is only of practical value for a fixed size array like str 30 name[20], where dimof(name) returns 20.
A clean way to remain in control, is to use a setter function:
static void TestArray(Args _args)
{
str 30 name[];
int n = 0;
int i;
void nameSet(int _i, str 30 _name)
{
n = max(n,_i);
name[_i] = _name;
}
;
nameSet(2,'abc');
nameSet(4,'def');
for (i = 1; i <= n; i++)
info(name[i]);
}
There is no upper bound index limit, so accessing name[7] is perfectly valid and in this case returns a blank value. This may be used to your advantage, if you always use all holes and never stores a blank:
static void TestArray(Args _args)
{
str 30 name[];
int i;
name[1] = 'abc';
name[2] = 'def';
for (i = 1; name[i]; i++)
info(name[i]);
}
Beware that accessing a higher index (in this case higher than 2) may in fact increase the allocated size of the array.