I'm trying to create a reader from a .car file (Content Addressed Archive) using #ipld/car. The documentation shows how to create a carReader when files are small as per below:
import { CarReader } from '#ipld/car'
const inStream = fs.createReadStream('example.car');
// read and parse the entire stream in one go, this will cache the contents of
// the car in memory so is not suitable for large files.
const reader = await CarReader.fromIterable(inStream);
Source
Question: If I have a file at C:\output.car, and it's say 10GB and larger than my available RAM, how can I create a reader for this large .car file?
I think it should be one of these methods, but there's no sample code on how to use it...
import { CarBlockIterator } from '#ipld/car/iterator';
// or
import { CarCIDIterator } from '#ipld/car/iterator';
Source
Figured it out. This works:
import { CarIndexedReader } from '#ipld/car';
const filename = "C:\output.car"
const carReader = await CarIndexedReader.fromFile(filename);
Related
I'm trying to load a large-ish (1000 lines, 68k) text file using
final String enString = await rootBundle.loadString('res/string/string_en.json');
The Dart class function AssetBundle.loadString that loads the string is
Future<String> loadString(String key, { bool cache = true }) async {
final ByteData data = await load(key);
if (data == null)
throw FlutterError('Unable to load asset: $key');
// 50 KB of data should take 2-3 ms to parse on a Moto G4, and about 400 μs
// on a Pixel 4.
if (data.lengthInBytes < 50 * 1024) {
return utf8.decode(data.buffer.asUint8List());
}
// For strings larger than 50 KB, run the computation in an isolate to
// avoid causing main thread jank.
return compute(_utf8decode, data, debugLabel: 'UTF8 decode for "$key"');
}
Looking at the code above, if the file is bigger than 50k, as mine is, an isolate is used.
As a test, I cut my file in half (so 32k) and it loaded in a second (not using the isolate). But, unedited, the function hangs when the isolate is used.
My files is just a simple json file of key-value pairs. Here are the first few lines
{
"ctaButtonConfirm": "Confirm",
"ctaButtonContinue": "Continue",
"ctaButtonReview": "Review",
"balance": "Balance",
"totalBalance": "Total Balance",
"transactions": "Transactions",
:
Seem like it hangs when the isolate is used?
EDIT
Based on the loadString code above I wrote an extension function that doesn't use an isolate and it works fine, so it's looking like the isolate doesn't like my file?
extension AssetBundleX on AssetBundle {
Future<String> loadStringWithoutIsolate(String key) async {
final ByteData data = await load(key);
return utf8.decode(data.buffer.asUint8List());
}
}
You can't access rootBundle from spawned isolate.
So use main isolate instead.
Or in [docs](This is useful for operations that take longer than a few milliseconds, and which would therefore risk skipping frames. For tasks that will only take a few milliseconds, consider SchedulerBinding.scheduleTask instead.)
you can try SchedulerBinding.scheduleTask instead.
How can I make possible that the app will load all of the images from the specific folder and then put in array and choose one image randomly? When chose one then pass to the fronted to show the image. How to do that too?
I am C# developer but not long time ago I found ElectronJS and this framework does everything easier so therefore I am moving to this framework.
I did in C# programming this way:
// basic settings.
var ext = new List<string> { ".jpg", ".gif", ".png" };
// we use same directory where program is.
string targetDirectory = Directory.GetCurrentDirectory() + "\\assets\\" + "images\\" + "animals\\";
// Here we create our list of files
// New list
// Use GetFiles to getfilenames
// Filter unwanted stuff away (like our program)
if (Directory.Exists(targetDirectory))
{
Files = new List<string>
(Directory.GetFiles(targetDirectory, "*.*", SearchOption.TopDirectoryOnly)
.Where(s => ext.Any(es => s.EndsWith(es))));
// Show first picture so we dont need wait 3 secs.
ChangePicture();
}
else
{
panel5.BackgroundImage = new Bitmap(Resources.doggy);
}
I don't know how to do in ElectronJS.
Thank you in advance the answers.
Alright. I found the solution.
However I don't understand the people who are giving negative reputation for the opened question. If they are giving negative reputation then they could explain why.
Well anyway, I did fix this issue with this way:
I created images.js file and added this:
var fs = require('fs');
function getRandImage() {
var files = fs.readdirSync('./assets/images/animals/')
/* now files is an Array of the name of the files in the folder and you can pick a random name inside of that array */
let chosenFile = files[Math.floor(Math.random() * files.length)]
console.log('../assets/images/animals/' + chosenFile);
return '../assets/images/animals/' + chosenFile;
}
module.exports = { getRandImage }
I used console to see if the value is correct, otherwise others can delete that part.
Sending the data to the renderer process:
const { getRandImage } = require('./images');
child.webContents.send('random-image', getRandImage());
I did put in the preload.js file the following (I used the starter pack electronjs github to start with something):
var { ipcRenderer } = require('electron');
ipcRenderer.on('random-image', function (event, store) {
document.getElementById("randompic").src = store;
console.log(store);
});
Same here, I did use console.log just for test the value is correct and I used to change the randompic ID related image src html to the randomly chosen image.
Hopefully I did helping those people who are newbie as me.
I'm trying to save a particular tab from a google spreadsheet into a folder and I need to schedule this to happen on the same day every month.
(I need to do that for 20 different sheets to create a backup every month.)
I've seen similar scripts converting to pdf or cvs but in my case I need to extract just one sheet and keep it as a google spreadsheet format inside a different folder.
Thank you for the help !
Answer
I've written a small piece of code that can handle your needs. I assume that all the files are in one folder (originalFilesFolderId) and you want to move them to another (copiedFilesFolderId). To achieve this goal you need to use:
SpreadsheetApp
DriveApp
Step by step
I have split the code in 4 functions:
The main function: It gets all the files from the originalFilesFolderId and calls process to start the manipulation.
process: it open the spreadsheet file, makes a copy and call the other two functions.
deleteSheets: it deletes all the sheets of the spreadsheet except the first one.
moveFile: it moves the copied file to the desired destination (copiedFilesFolderId)
Code
function main() {
var files = DriveApp.getFolderById('originalFilesFolderId').getFiles()
while (files.hasNext()) {
var file = files.next();
var fileId = file.getId()
process(fileId)
}
}
function process(fileId) {
try {
var ss = SpreadsheetApp.openById(fileId)
var copied = ss.copy("copy of " + ss.getName())
moveFile(copied)
deleteSheets(copied)
}
catch (err) {
console.log(err.message)
}
}
function deleteSheets(copied) {
var sheets = copied.getSheets()
for (var i = 1; i < sheets.length; i++) {
copied.deleteSheet(sheets[i])
}
}
function moveFile(copied) {
var copiedId = copied.getId()
var file = DriveApp.getFileById(copiedId)
file.moveTo('copiedFilesFolderId')
}
References
SpreadsheetApp
DriveApp
So I'm trying to cache an image if an upload fails, due to the current limitations of flutter I think I will have to save it to shared preferences as a base64 file, then get it from shared preferences, convert it back to an image then upload that to firebase storage. My current code looks like so:
void saveImageToCache(File image) async {
List<int> imageBytes = image.readAsBytesSync();
String base64Image = base64Encode(imageBytes); //convert image ready to be cached as a string
var cachedImageName = "image $fileName";
instance.setString(cachedImageName, base64Image); // set image name in shared preferences
var retrievedImage = instance.getString(cachedImageName);// once a connection has been established again, get the image file from the cache and send it to firebase storage as an image
storageReference.putData(retrievedImage, StorageMetadata(contentType: 'base64'));
var prefix = "data:image/png;base64,";
var bStr = retrievedImage.substring(prefix.length);
var bs = Base64Codec.codec.decodeString(bStr);
var file = new File("image.png");
file.writeAsBytesSync(bs.codeUnits);
uploadTask = storageReference.child(fileName).putFile(file, const StorageMetadata(contentLanguage: "en"));
}
This is failing for me at var bStr = retrievedImage.substring(prefix.length); with error type 'String' is not a subtype of type 'Uint8List' where and im still not sure if im doing the right thing.
Any help would be great thanks.
I wouldn't recommend storing binary files to shared preferences. Especially since you're building an image cache.
I'd just store them to a file.
Future<File> saveFile(File toBeSaved) async {
final filePath = '${(await getApplicationDocumentsDirectory()).path}/image_cache/image.jpg';
File(filePath)
..createSync(recursive: true)
..writeAsBytes(toBeSaved.readAsBytesSync());
}
This uses getApplicationDocumentsDirectory() from the path provider package.
Can we use dart to download a file?
For example in python
I'm using the HTTP package a lot. If you want to download a file that is not huge, you could use the HTTP package for a cleaner approach:
import 'package:http/http.dart' as http;
main() {
http.get(url).then((response) {
new File(path).writeAsBytes(response.bodyBytes);
});
}
What Alexandre wrote will perform better for larger files. Consider writing a helper function for that if you find the need for downloading files often.
Shailen's response is correct and can even be a little shorter with Stream.pipe.
import 'dart:io';
main() async {
final request = await HttpClient().getUrl(Uri.parse('http://example.com'));
final response = await request.close();
response.pipe(File('foo.txt').openWrite());
}
The python example linked to in the question involves requesting the contents of example.com and writing the response to a file.
Here is how you can do something similar in Dart:
import 'dart:io';
main() {
var url = Uri.parse('http://example.com');
var httpClient = new HttpClient();
httpClient.getUrl(url)
.then((HttpClientRequest request) {
return request.close();
})
.then((HttpClientResponse response) {
response.transform(new StringDecoder()).toList().then((data) {
var body = data.join('');
print(body);
var file = new File('foo.txt');
file.writeAsString(body).then((_) {
httpClient.close();
});
});
});
}
We can use http.readBytes(url).
await File(path).writeAsBytes(await http.readBytes('https://picsum.photos/200/300/?random'));
Yes, first of all you have to request to file url using http dart library like:
Response response = await get(Uri.parse(link));
after that your Response object (response) will get that file in self and you can simply write the response bytes to a file and that file will be your downloaded file.
as I open file like this:
File file = File('image.jpg')
then we have to send response bytes to this file like this:
file.writeAsBytes(response.bodyBytes);
now you have downloaded a image file successfully.. Congrates.
additional, for example let me show you a sample code to download a image file :
import 'dart:io';
import 'package:http/http.dart';
main(List<String> args) async {
var link =
"https://pps.whatsapp.net/v/t61.24694-
24/72779382_449683642563635_3243701117464346624_n.jpg?ccb=11-
4&oh=23e3bc2ce3f4940a70cb464494bbda76&oe=619B3B8C";
Response response = await get(Uri.parse(link));
File file = File('image.jpg');
file.writeAsBytes(response.bodyBytes);
}
look, this is the code and a file named image.jpg is downloaded at bottom in terminal view is our downloaded image.
screen shot
this is our actual image which we downloaded.
downloaded image