Call libc function from JNA - jna

I use a C library from Java through JNA and one function does not flush properly (since the output appear all at once on program end). I have tried Java side System.out.flush(); with no luck.
In brief, I would like to call C fflush(stdout) from Java. With JNA already there (thus would prefer if no additional library) and without C to write.
I am aware of JNA Library mapping as in this question but that seems overkill to me.

The JNA library wrapping way code is actually not so heavy (at least for the flush all behavior).
protected interface CLibrary extends Library
{
static CLibrary clib = (CLibrary) Native.loadLibrary ("c", CLibrary.class);
int fflush (Pointer stream);
}
/* ... */
CLibrary.clib.fflush (null);
JNA also offer late binding method and these oneliners will do what you want
NativeLibrary.getInstance ("c").getFunction ("fflush").invokeInt (new Object[]{0});
// even shorter
Function.getFunction ("c", "fflush").invokeInt (new Object[]{0});
The tedious part comes when you want to limit flushing to stdout. You have to deal with vendor-specific code (stdout is either defined as a macro expanding to an array, Amtel avr-libc, to a function call, Microsoft msvcrt, or a pointer in GNU libc).
For the libc, you might use (two lines for legibility)
Pointer stdout = NativeLibrary.getInstance ("c").getGlobalVariableAddress ("stdout").getPointer (0);
Function.getFunction ("c", "fflush").invokeInt (new Object[]{stdout});

Adding this answer for Win32 / Win64 users, complementing FabienAndre's for GNU libc.
Selectively flushing the stdout stream calling the system's c library's fflush method via jna is hard and cumbersome. As FabienAndre already mentioned, it is difficult to get a hold of the stdout macro definition. For msvcrt (the Win32 / Win64 C library) it is defined via a function call to __iob_func(); the latter returning a pointer to an array of FILE structures. At index 0 is stdin, index 1 is stdout and index 2 is stderr. So for flushing stdout you even need to know the size of the FILE structure, of course, it is different for Win32 and Win64 ...
The following example is tested under Win64 but ought to work under Win32. It was inspired by the thread JNA solutions to catch stdout/stderr of DLL.
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Platform;
import com.sun.jna.Pointer;
public class JnaTest {
public interface CLibrary extends Library {
CLibrary INSTANCE = (CLibrary) Native.loadLibrary("msvcrt" , CLibrary.class);
Pointer __iob_func();
void printf(String format, Object... args);
int fflush (Pointer stream);
}
public static void main(String[] args) {
int sizeOfFileStructure = Platform.is64Bit() ? 48 : 32;
Pointer stdout = CLibrary.INSTANCE.__iob_func().share(sizeOfFileStructure);
CLibrary.INSTANCE.printf("Hello, World\n");
CLibrary.INSTANCE.fflush(stdout);
}
}

Related

How can I get a custom python type and avoid importing a python module every time a C function is called

I am writing some functions for a C extension module for python and need to import a module I wrote directly in python for access to a custom python type. I use PyImport_ImportModule() in the body of my C function, then PyObject_GetAttrString() on the module to get the custom python type. This executes every time the C function is called and seems like it's not very efficient and may not be best practice. I'm looking for a way to have access to the python custom type as a PyObject* or PyTypeObject* in my source code for efficiency and I may need the type in more than one C function also.
Right now the function looks something like
static PyObject* foo(PyObject* self, PyObject* args)
{
PyObject* myPythonModule = PyImport_ImportModule("my.python.module");
if (!myPythonModule)
return NULL;
PyObject* myPythonType = PyObject_GetAttrString(myPythonModule, "MyPythonType");
if (!myPythonType) {
Py_DECREF(myPythonModule);
return NULL;
}
/* more code to create and return a MyPythonType instance */
}
To avoid retrieving myPythonType every function call I tried adding a global variable to hold the object at the top of my C file
static PyObject* myPythonType;
and initialized it in the module init function similar to the old function body
PyMODINIT_FUNC
PyInit_mymodule(void)
{
/* more initializing here */
PyObject* myPythonModule = PyImport_ImportModule("my.python.module");
if (!myPythonModule) {
/* clean-up code here */
return NULL;
}
// set the static global variable here
myPythonType = PyObject_GetAttrString(myPythonModule, "MyPythonType");
Py_DECREF(myPythonModule);
if (!myPythonType) {
/* clean-up code here */
return NULL;
/* finish initializing module */
}
which worked, however I am unsure how to Py_DECREF the global variable whenever the module is finished being used. Is there a way to do that or even a better way to solve this whole problem I am overlooking?
First, just calling import each time probably isn't as bad as you think - Python does internally keep a list of imported modules, so the second time you call it on the same module the cost is much lower. So this might be an acceptable solution.
Second, the global variable approach should work, but you're right that it doesn't get cleaned up. This is rarely a problem because modules are rarely unloaded (and most extension modules don't really support it), but it isn't great. It also won't work with isolated sub-interpreters (which isn't much of a concern now, but may become more more popular in future).
The most robust way to do it needs multi-phase initialization of your module. To quickly summarise what you should do:
You should define a module state struct containing this type of information,
Your module spec should contain the size of the module state struct,
You need to initialize this struct within the Py_mod_exec slot.
You need to create an m_free function (and ideally the other GC functions) to correctly decref your state during de-initialization.
Within a global module function, self will be your module object, and so you can get the state with PyModule_GetState(self)

How to see dart core library code (StringBuffer)?

I just wondered how the StringBuffer's concrete code is look like.
But I can only find abstract method like this. Well...
Could you please let me know how to get to the concrete code ? Thanks!
// Copyright (c) 2011, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
part of dart.core;
class StringBuffer implements StringSink {
/// Creates a string buffer containing the provided [content].
external StringBuffer([Object content = ""]);
/// Returns the length of the content that has been accumulated so far.
/// This is a constant-time operation.
external int get length;
/// Returns whether the buffer is empty. This is a constant-time operation.
bool get isEmpty => length == 0;
/// Returns whether the buffer is not empty. This is a constant-time
/// operation.
bool get isNotEmpty => !isEmpty;
/// Adds the string representation of [object] to the buffer.
external void write(Object? object);
/// Adds the string representation of [charCode] to the buffer.
///
/// Equivalent to `write(String.fromCharCode(charCode))`.
external void writeCharCode(int charCode);
/// Writes all [objects] separated by [separator].
///
/// Writes each individual object in [objects] in iteration order,
/// and writes [separator] between any two objects.
external void writeAll(Iterable<dynamic> objects, [String separator = ""]);
external void writeln([Object? obj = ""]);
/// Clears the string buffer.
external void clear();
/// Returns the contents of buffer as a single string.
external String toString();
}
It depends on what target platform you are interested in which is also the reason why the do this since the external function can point to different implementation depending on target.
The implementation for the different platforms can be found here:
https://github.com/dart-lang/sdk/tree/2.18.1/sdk/lib/_internal
So if you want to see the implementation used when running the Dart VM or compiled to AOT binary, the implementation of StringBuffer can be found here:
https://github.com/dart-lang/sdk/blob/2.18.1/sdk/lib/_internal/vm/lib/string_buffer_patch.dart
Another point here is that Dart VM code often have stuff like the following which you can see in the bottom of the StringBuffer implementation:
#pragma("vm:external-name", "StringBuffer_createStringFromUint16Array")
external static String _create(Uint16List buffer, int length, bool isLatin1);
This means that we do a call to the C++ code in the Dart runtime (which is bundled together with your compiled app). So the _create call will end up calling the following method:
https://github.com/dart-lang/sdk/blob/2.18.1/runtime/lib/string.cc#L516-L534
If your target are instead JavaScript, the code used for that implementation (when compiled to a release ready JS bundle) can be found here:
https://github.com/dart-lang/sdk/blob/2.18.1/sdk/lib/_internal/js_runtime/lib/core_patch.dart#L632

FindWindowA function does not yield the expected output in dart foreign function interface

The following dart code's expected output is the handle to the file explorer window but the output always comes out to be 0 even though the file explorer window does exist!
import 'package:ffi/ffi.dart';
import 'dart:ffi';
void main() {
DynamicLibrary dl = DynamicLibrary.open('user32.dll');
final func = dl.lookupFunction<
IntPtr Function(Pointer<Utf16>?, Pointer<Utf16>?),
int Function(Pointer<Utf16>?, Pointer<Utf16>?)
>('FindWindowA');
print(func(nullptr, 'File Explorer'.toNativeUtf16()));
}
I have ran the function FindWindowA in a c++ program and it has returned the exptected output with the same input values that are NULL and 'File Explorer'.
In dart using null throws the error Invalid argument(s): argument value for ':ffi_param1' is null; hence the use of nullptr
FindWindowA does not take pointers to UTF-16 strings. The A in FindWindowA stands for "ANSI", which is Microsoft's way of indicating that it uses strings with single-byte code units encoded in the system's local encoding. (This is also indicated by its signature, which takes LPCSTR arguments.)
If you want to pass UTF-16 strings (which in Microsoft Windows are considered "wide strings"), then use FindWindowW.

Z3 API: Crash when parsing fixed point SMTLib string

I am trying to use the C/C++ API of Z3 to parse fixed point constraints in the SMTLib2 format (specifically files produced by SeaHorn). However, my application crashes when parsing the string (I am using the Z3_fixedpoint_from_string method). The Z3 version I'm working with is version 4.5.1 64 bit.
The SMTLib file I try to parse works find with the Z3 binary, which I have compiled from the sources, but it runs into a segmentation fault when calling Z3_fixedpoint_from_string. I narrowed the problem down to the point that I think the issue is related to adding relations to the fixed point context. A simple example that produces a seg fault on my machine is the following:
#include "z3.h"
int main()
{
Z3_context c = Z3_mk_context(Z3_mk_config());
Z3_fixedpoint f = Z3_mk_fixedpoint(c);
Z3_fixedpoint_from_string (c, f, "(declare-rel R ())");
Z3_del_context(c);
}
Running this code with valgrind reports a lot of invalid reads and writes. So, either this is not how the API is supposed to be used, or there is a problem somewhere. Unfortunately, I could not find any examples on how to use the fixed point engine programmatically. However, calling Z3_fixedpoint_from_string (c, f, "(declare-var x Int)"); for instance works just fine.
BTW, where is Z3_del_fixedpoint()?
The fixedpoint object "f" is reference counted. the caller is responsible for taking a reference count immediately after it is created. It is easier to use C++ smart pointers to control this, similar to how we control it for other objects. The C++ API does not have a wrapper for fixedpoint objects so you would have to create your own in the style of other wrappers.
Instead of del_fixedpoint one uses reference counters.
class fixedpoint : public object {
Z3_fixedpoint m_fp;
public:
fixedpoint(context& c):object(c) { mfp = Z3_mk_fixedpoint(c); Z3_fixedpoint_inc_ref(c, m_fp); }
~fixedpoint() { Z3_fixedpoint_dec_ref(ctx(), m_fp); }
operator Z3_fixedpoint() const { return m_fp; }
void from_string(char const* s) {
Z3_fixedpoint_from_string (ctx(), m_fp, s);
}
};
int main()
{
context c;
fixedpoint f(c);
f.from_string("....");
}

Switching from Rhino to Nashorn

I have a Java 7 project which makes a lot of use of Javascript for scripting various features. Until now I was using Rhino as script engine. I would now like to move to Java 8, which also means that I will replace Rhino by Nashorn.
How compatible is Nashorn to Rhino? Can I use it as a drop-in replacement, or can I expect that some of my scripts will not work anymore and will need to be ported to the new engine? Are there any commonly-used features of Rhino which are not supported by Nashorn?
One problem is that Nashorn can no longer by default import whole Java packages into the global scope by using importPackage(com.organization.project.package);
There is, however, a simple workaround: By adding this line to your script, you can enable the old behavior of Rhino:
load("nashorn:mozilla_compat.js");
Another problem I ran into is that certain type-conversions when passing data between java and javascript work differently. For example, the object which arrives when you pass a Javascript array to Java can no longer be cast to List, but it can be cast to a Map<String, Object>. As a workaround you can convert the Javascript array to a Java List in the Javascript code using Java.to(array, Java.type("java.util.List"))
To use the importClass method on JDK 8, we need to add the following command:
load("nashorn:mozilla_compat.js");
However, this change affect the execution on JDK 7 (JDK does not gives support to load method).
To maintain the compatibility for both SDKs, I solved this problem adding try/catch clause:
try{
load("nashorn:mozilla_compat.js");
}catch(e){
}
Nashorn can not access an inner class when that inner class is declared private, which Rhino was able to do:
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
public class Test {
public static void main(String[] args) {
Test test = new Test();
test.run();
}
public void run() {
ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("JavaScript");
Inner inner = new Inner();
engine.put("inner", inner);
try {
engine.eval("function run(inner){inner.foo(\"test\");} run(inner);");
} catch (ScriptException e) {
e.printStackTrace();
}
}
private class Inner {
public void foo(String msg) {
System.out.println(msg);
}
}
}
Under Java8 this code throws following exception:
javax.script.ScriptException: TypeError: kz.test.Test$Inner#117cd4b has no such function "foo" in <eval> at line number 1
at jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:564)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:548)
I noticed that Rhino didn't have a problem with a function called 'in()' (although 'in' is a reserved JavaScript keyword).
Nashorn however raise an error.
Nashorn cannot call static methods on instances! Rhino did this, therefore we had to backport Rhino to Java 8 (Here's a short summary: http://andreas.haufler.info/2015/04/using-rhino-with-java-8.html)
Nashorn on Java8 does not support AST. So if you have Java code that inspects the JS source tree using Rhino's AST mechanism , you may have to rewrite it (using regex maybe) once you port your code to use Nashorn.
I am talking about this API https://mozilla.github.io/rhino/javadoc/org/mozilla/javascript/ast/AstNode.html
Nashorn on Java9 supports AST though.
One feature that is in Rhino and not Nashorn: exposing static members through instances.
From http://nashorn-dev.openjdk.java.narkive.com/n0jtdHc9/bug-report-can-t-call-static-methods-on-a-java-class-instance : "
My conviction is that exposing static members through instances is a
sloppy mashing together of otherwise separate namespaces, hence I
chose not to enable it.
I think this is deeply wrong. As long as we have to use two different constructs to access the same java object and use package declarations unnecessarily in javascript, code becomes harder to read and write because cognitive load increases. I will rather stick to Rhino then.
I have not found a workaround for this obvious "design bug" yet.

Resources