I'm currently developing a notification plugin to be used with my music application that I'm converting over to Flutter. Thus far it's all been working perfectly, however though I ran into a problem which I'm not sure how to handle.
My plugin requires an image which is displayed in the notification, The images are all hosted and fetched via url (https://example.com/img.png) so that eliminates the loading via file system
Now the problem is that, I would like to keep the plugin as lightweight as possible (would rather not add Glide etc).
Is there anyway I can pass the bitmap directly from flutter to the plugin ? perhaps the same way that we can pass strings ?
static Future example(String data) async {
await _channel.invokeMethod('example', {
'data' : data
});
}
Thanks in advance guys, Any advice is appreciated.
The StandardMessageCodec which converts between Dart and native types handles
acyclic values of these forms:
null
[bool]s
[num]s
[String]s
[Uint8List]s, [Int32List]s, [Int64List]s, [Float64List]s
[List]s of supported values
[Map]s from supported values to supported values
so you can pass a Uint8List (in place of String data) in your example, and you will get a byte[] or FlutterStandardTypedData on the native side for Android and iOS respectively.
It's not clear whether you need the png on the native side or the decoded bitmap.
To fetch the png you could use the http package
import 'dart:typed_data';
import 'package:http/http.dart';
Uint8List png =
(await get('http://www.barcodes4.me/barcode/c39/123456.png')).bodyBytes;
and pass Uint8List png to invokeMethod. If you need to convert to a bitmap first (though you should avoid this if you can, as the bitmap will be much larger than the png), use the image package.
import 'package:image/image.dart';
Image image = decodeImage(png); // if known to be a PNG, could call decodePng
Map<String, dynamic> imageData = {
'width': image.width,
'height': image.height,
'bitmap': image.getBytes(),
};
and pass Map imageData to invokeMethod. This will appear at the native end as a java.util.HashMap or NSDictionary.
Related
how do i convert a http response to a buffer? im using the http pub package https://pub.dev/packages/http and one of the endpoints of an api im using is returning an image file, i want to convert it to a buffer. in JS i'd just do
const result = await res.buffer();
but how do i do it in dart?
I tried a few different methods of the Response class but couldnt get my head around it
Node describes Buffer as
The Buffer class is a subclass of JavaScript's Uint8Array class
So it may safe assume that you can use the Response.bodyBytes (which is a dart Uint8List type) field as jamesdlin suggested.
In Dart, how do I get from the AST to Elements?
I generate the AST like this:
var ast = parseFile(path: "/home/user/myFile.dart", featureSet: FeatureSet.latestLanguageVersion());
I can access the declaration nodes, but I would like get to the Elements (e.g. LibraryElement, ClassElement, etc.).
How to do this?
The Dart Analyzer is built to work with a filesystem structure, and cannot build an element tree from a single file alone.
The analysis APIs to analyze a collection of files are described in this document.
On another note, the analyzer package also provides APIs to emulate a filesystem in-memory.
This means that library code in the form of a String can also be analyzed if need be, by emulating a file.
In order to analyse a single file, we can use these APIs alongside, if need be, an in-memory filesystem structure layered on top of the real filesystem, to allow the analyzer to access required SDK files in addition to the emulated file.
import 'package:analyzer/dart/analysis/analysis_context_collection.dart';
import 'package:analyzer/dart/analysis/results.dart';
import 'package:analyzer/file_system/overlay_file_system.dart';
import 'package:analyzer/file_system/physical_file_system.dart';
const filePath = '/path/to/myFile.dart';
final collection = AnalysisContextCollection(
includedPaths: const [filePath],
// If using an in-memory file, also provide a layered ResourceProvider:
resourceProvider: OverlayResourceProvider(PhysicalResourceProvider())
..setOverlay(
filePath,
content: libraryCode,
modificationStamp: 0,
),
);
final analysisSession = collection.contextFor(filePath).currentSession;
final libraryElement = await analysisSession
.getLibraryByUri('file://$filePath')
.then((libraryResult) => (libraryResult as LibraryElementResult).element);
I would like to include a Kotlin file that only performs data processing and network operations in an existing iOS project, while keeping native iOS UI code.
While I thought that this may be achievable with Kotlin/Native, the iOS samples (1,2) that I found that use Kotlin/Native seem to take over the iOS UI code as well.
Is including a Kotlin file for data transfer in iOS possible with Kotlin/Native without touching the UI code, and if so, what are the steps to do so?
Yes, it is possible in a cross-platform project to transfer data between Kotlin and native iOS UI Code by using Kotlin/Native. This allows to have a common code base for the data model based on Kotlin, while e.g. continuing to use native UI code for iOS.
The original proof:
The project https://github.com/justMaku/Kotlin-Native-with-Swift pointed me in the right direction, since it shows the essential steps to do so:
In a Swift UIViewController, it calls a wrapper function that shall receive a string from a Kotlin function. The call is mediated through a C++ layer, which itself starts the Kotlin runtime, passes the request to a Kotlin function, receives the string from it, and passes it back to the Swift UIViewController, which then displays it.
On the technical level, the project contains a script that compiles the Kotlin, C++, and Kotlin/Native part into a static library, which then can be called from the native iOS project.
To get the code to run, I had (after cloning from git) to perform a "git submodule sync" before running "./setup.sh".
To transfer data with a data model based on Kotlin, I would like to have a generic function, that can pass data to Kotlin, modify that data, and return the result back to the native iOS code. As a proof of principle, that such a function can be build, I extended the project to not only receive a string from Kotlin, but send one to Kotlin, append it, and send the result back.
Extension of the project:
Since there were some roadblocks in this seemingly simple extension, I lay out the steps for anybody interested. If you follow along, you should get the following displayed:
The text may be stupid, but it tells you, what happens.
The changes in ViewController.swift in the function viewDidAppear are:
let swiftMessage: String = "Hello Kotlin, this is Swift!"
let cStr = swiftMessage.cString(using: String.Encoding.utf8)
if let retVal = kotlin_wrapper(cStr) {
let string = String(cString: retVal)
...
}
You see the text that Swift sends to Kotlin in the wrapper function (in the end, the resulting 'string' variable will be displayed). One could directly pass the Swift String to the wrapper, but I wanted to highlight that the wrapper will consider the input and output as c-strings. Indeed, the file Kotlin Native-Bridging-Header.h inside the native iOS project now becomes:
extern const char* kotlin_wrapper(const char* swiftMessage);
On it goes to the file Launcher.cpp. Since the original file used a KString as result value of kotlin_main, I tried for some time to convert const char* to KString and pass that to kotlin_main. In the end I found, that it is much simpler to directly transfer the const char* variables to Kotlin, and do the transformation there with the functions that are given to us by Kotlin/Native.
My Launcher.cpp then became more compact than the original. Here is the complete file:
#include "Memory.h"
#include "Natives.h"
#include "Runtime.h"
#include "KString.h"
#include <stdlib.h>
#include <string>
extern "C" const char* kotlin_main(const char* swiftMessageChar);
extern "C" const char* kotlin_wrapper(const char* swiftMessageChar) {
RuntimeState* state = InitRuntime();
if (state == nullptr) {
return "Failed to initialize the kotlin runtime";
}
const char* exitMessage = kotlin_main(swiftMessageChar);
DeinitRuntime(state);
return exitMessage;
}
You see how the wrapper first starts the Kotlin runtime and then calls the function kotlin_main, which resides in the file kotlin.kt:
import konan.internal.ExportForCppRuntime
import kotlinx.cinterop.CPointer
import kotlinx.cinterop.ByteVar
import kotlinx.cinterop.cstr
import kotlinx.cinterop.nativeHeap
import kotlinx.cinterop.toKString
#ExportForCppRuntime
fun kotlin_main(cPtr: CPointer<ByteVar>): CPointer<ByteVar> {
val swiftMessage = cPtr.toKString()
val kotlinMessage = "Hello Swift, I got your message: '$swiftMessage'."
val returnPtr = kotlinMessage.cstr.getPointer(nativeHeap)
return returnPtr
}
The pointer is converted to a Kotlin String, and then used in the creation of the kotlinMessage (the example of a data transformation). The result message is then transformed back to a pointer, and passed through the wrapper back to the Swift UIViewController.
Where to go from here?
In principle, one could use this framework without touching the C++ layer again. Just define pack and unpack functions, that pack arbitrary data types into a string and unpack the string to the respective data type on the other side. Such pack and unpack functions have to be written only once per language, and can be reused for different projects, if done sufficiently generic. In practice, I probably would first rewrite the above code to pass binary data, and then write the pack and unpack functions to transform arbitrary data types to and from binary data.
You can use kotlin as a framework if you want, so the kotlin code stays in framework file so you can use some common code on both android and iOS without writing your complete iOS app in kotlin.
Use gradle to build your kotlin code in objc/swift compatible framework
In your build.gradle file
buildscript {
ext.kotlin_native_version = '0.5'
repositories {
mavenCentral()
maven {
url "https://dl.bintray.com/jetbrains/kotlin-native-dependencies"
}
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:$kotlin_native_version"
}
}
group 'nz.salect'
version '0.1'
apply plugin: "konan"
konan.targets = ["iphone", "iphone_sim"]
konanArtifacts {
framework('nativeLibs')
}
It will generate two .framework files, one for simulator other for the actual device, put the framework in your project and link that to your project as any other third party framework.
Cmd: ./gradlew build
Note: Every time you change your kotlin files build and replace your
framework file as well(you can create a shell script and add that to
build phases to do that automatically).
Cheers !!!
I'm trying to figure out how to convert an image from a stream with ImageResizer (http://imageresizing.net/).
I have tried something like this.
Stream s = WebRequest.Create("http://example.com/resources/gfx/unnamed.webp").GetResponse().GetResponseStream();
ImageBuilder.Current.Build(s, "~/resources/gfx/photo3.png", new ResizeSettings("format=png"));
But i just get the error
"File may be corrupted, empty, or may contain a PNG image with a single dimension greater than 65,535 pixels."
When i do
using (Stream output = File.OpenWrite(Server.MapPath("~/resources/gfx/test.webp")))
using (Stream input = WebRequest.Create("http:///example.com/resources/gfx/unnamed.webp").GetResponse().GetResponseStream()) {
input.CopyTo(output);
}
ImageBuilder.Current.Build("~/resources/gfx/test.webp", "~/resources/gfx/photo3.png",
new ResizeSettings("format=png"));
It works fine, am i'm missing something here?
It's possible that 'output' has not been flushed to disk. .NET 4+ doesn't guarantee the file's actually written to disk just because you disposed the stream.
I assume you have the ImageResizer.Plugins.WebP plugin installed?
As part of my project I wanted to send stream of images using websockets from embedded machine to client application and display them in img tag to achieve streaming.
Firstly I tried to send raw RGB data (752*480*3 - something about 1MB) but in the end I got some problems with encoding image to png in javascript based on my RGB image so I wanted to try to encode my data to PNG firstly and then sent it using websockets.
The thing is, I am having some problems with encoding my data to PNG using OpenCV library that is already used in the project.
Firstly, some code:
websocketBrokerStructure.matrix = cvEncodeImage(0, websocketBrokerStructure.bgrImageToSend, 0);
websocketBrokerStructure.imageDataLeft = websocketBrokerStructure.matrix->rows * websocketBrokerStructure.matrix->cols * websocketBrokerStructure.matrix->step;
websocketBrokerStructure.imageDataSent = 0;
but I am getting strange error during execution of the second line:
terminate called after throwing an instance of 'std::logic_error'
what(): basic_string::_S_construct NULL not valid
and I am a bit confused why I am getting this error from my code.
Also I am wondering if I understand it right: after invoking cvEncodeImage (where bgrImage is IplImage* with 3 channels - BGR) I just need to iterate through data member of my CvMatto get all of the png encoded data?
The cvEncodeImage function takes as its first parameter the extension of the image you want to encode. You are passing 0, which is the same thing as NULL. That's why you are getting the message NULL not valid.
You should probably use this:
websocketBrokerStructure.matrix = cvEncodeImage(".png", websocketBrokerStructure.bgrImageToSend, 0);
You can check out the documentation of cvEncodeImage here.
You can check out some examples of cvEncodeImage, or its C++ brother imencode here: encode_decode_test.cpp. They also show some parameters you can pass to cvEncodeImage in case you want to adjust them.