PInvoke Unbalances the stack - stack

I have a c++ function to be exported as a .dll to use:
static __declspec(dllexport) void DiagEncrypt(
UCHAR * ptrDataByte,
size_t numBytes,
ACCESS_LEVEL accLevel);
void DiagnosticCrypto::DiagEncrypt(UCHAR * ptrDataByte, size_t numBytes, ACCESS_LEVEL accLevel)
And I import it in my C# program:
[DllImport("DiagnosticCryptoDll.dll", EntryPoint = "?DiagEncrypt#DiagnosticCrypto#DiagnosticCryptoDll##SAXPAEIW4ACCESS_LEVEL#12##Z", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
static extern void DiagEncrypt(
//[MarshalAs(UnmanagedType.LPArray)] byte[] data,
IntPtr data,
uint numBytes, //UintPtr numBytes,
ACCESS_LEVEL accLevel);
when I execute it, there's an error of unbalanced stack.
Can somebody help me to figure out where the error is?
The marked part is what I have tried but it failed.

Your C++ code uses the cdecl calling convention but your p/invoke uses stdcall. Change or the other to match.
For example, you could change the p/invoke to use CallingConvention.Cdecl.
The other comments I would make are:
It may be preferable to use extern "C" to suppress C++ name mangling. Although in the comments, Hans argues for mangling.
You've mapped size_t to uint. That will work fine on 32 bit, but not on 64 bit, where size_t is 64 bits wide. I suggest that you use UIntPtr here.
Are you quite sure that the function calls SetLastError. It's not a Windows function so I'd guess it does not.

Related

File operations in drivers

I was trying to figure out how file operations in drivers work. I know there are several file operations but the functions for these operations are called with several arguments while the operations themselves are defined without any.
So if I have this -
static const struct file_operations proc_myled_operations = {
.open = proc_myled_open,
.read = seq_read,
.write = proc_myled_write,
.llseek = seq_lseek,
.release = single_release
};
Now I know that kernel level drivers can only be accessed as files from the user application. This is an embedded system so I have some LEDs that I can turn on by writing to their memory mapped registers.
So the .write or "proc_myled_write" call will execute when I turn an led on which I can do by opening this file using fopen and then writing to it by using fputs. But if .write is mapped as "proc_myled_write and this function has arguments like so -
static ssize_t proc_myled_write(struct file *file, const char __user * buf,
size_t count, loff_t * ppos)
What happens to the arguments? There is no function call for the above function with those arguments. I've seen this in several drivers. I just used this one because it was a simple example. How are the file operations mapped to these functions? How does the, for example, "write" in user space trace to the write in the driver?
Thank you.
I'm not exactly sure what you mean when you say "There is no function call for the above function with those arguments."
The prototype for these functions is defined in the declaration for struct file_operations itself.
Here is the first few lines from the struct declaration:
struct file_operations {
struct module *owner;
loff_t (*llseek) (struct file *, loff_t, int);
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
...
While the arguments are not named in the declaration, you can clearly see that the write() function is declared with 4 parameters matching the types that you mention in your question.
When you assign the function to its appropriate field (proc_myled_operations.write = proc_myled_write) you are simply passing a pointer to the write function declared and defined in your module. Pointers to functions themselves do not need parameters.
Ok, so you're question really is: "How does the user space system call eventually call the write function in your module?" Good question! I recommend editing your question to make that clearer for future readers.
Well, let's see if I can follow the paper trail. I discovered this document to give me the starting location to look in the code for the write() system call. It's very very old, but hey, not everything changes in the kernel! We start our journey at the write() system call declaration in fs/read_write.c:
SYSCALL_DEFINE3(write, unsigned int, fd, const char __user *, buf,
size_t, count)
It uses the file descriptor fd to get the struct file created when you registered your character driver. Then it gets the current position in the file and calls vfs_write().
ssize_t vfs_write(struct file *file, const char __user *buf, size_t count, loff_t *pos)
And it is in this function see the following line:
ret = file->f_op->write(file, buf, count, pos);
There it is!
To allay any doubts as to the type of file->f_op, we take a look at the definition of struct file and see the following definition for the f_op field:
const struct file_operations *f_op;
So it must be the struct file_operations you passed in when you registered your driver. Phew!
Hopefully all of these links will show you how to follow the trail for other system calls if you are curious.
#Maverick, write prototype or signature in kernel is ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *). From user space application you will be issuing open, write system call to turn on/off the LED. Write system call signature in user space is int write(int fd, const char *buf, size_t count). So when you call write from user space, fd (file descriptor) passed reaches the virtual file system(vfs), maintains linked list of the open file descriptor table(OFDT), thus according to the fd, OFDT has an pointer filp (file pointer) which point to the file opened for ex: device node "/dev/xxx" or any other file for that matter. The rest buf and count are same arguments passed from the user space to the kernel space. Last loff_t *fpos comes into picture if you want to seek the file for ex: using lseek or fseek on the file opened, if seek then the file pointer(loff_t fpos) position gets changed accordingly. Hope I cleared your doubt :-)

Luabind calling convention issues

I am having an issue with Luabind that I am unsure of how to fix without some over-simplified solution.
Luabind appears to only allow binding to functions using the __cdecl calling convention. In my current project all of the functionality exposed to extensions/plugins is exposed using __stdcall. This leaves me unable to bind the exposed objects directly and instead I have to make wrappers for the objects exposed. This would be fine but there are a lot of objects that would need to be wrapped.
For example, an object can look like this:
struct IObject
{
void __stdcall SomeFunc1( void );
void __stdcall SomeFunc2( const char* );
};
struct IObjectContainer
{
IObject* __stdcall GetObject( int );
IObject* __stdcall GetObject( const char* );
};
struct IObjectCore
{
IObjectContainer* __stdcall GetObjectContainer();
};
I don't have the option of changing the entire projects calling convention currently so I am seeing if someone has a solution to perhaps patch Luabind to work with __stdcall functions. I am not the best with templates and with boost things, so I'm personally unsure where to even start trying to add the ability to use __stdcall functions.
For reference, I am using:
Lua 5.1.4
Luabind 0.9.1
VS2010
Both Lua and Luabind are stock latest versions of their rev. (Not using Lua 5.2 for project restriction reasons, but if there is a __stdcall fix for 5.2/Luabind I will gladly take that as well.)
I could only find a fix for a very old version of Luabind to do this but the patch floating on the net still for that does not line up with the current Luabind code at all.
If there is any other information needed feel free to ask.
Sadly due to inactivity and no further answers from more searching I spoke with the project developer and have gotten the entire project stripped of __stdcall. So the bindings all work fine now via __cdecl. Not the route I wanted to take but things are working as planned now.
I faced the exact same problem when binding OpenGL (with GLEW functions) to Lua, and solved it using variadic templates.
Now if the function is global and you know its address in compile time, you can be good with something like this:
template<typename Signature>
struct wrap_known;
template<typename Ret, typename... Args>
struct wrap_known<Ret __stdcall (Args...)> {
template <Ret __stdcall functor(Args...)>
static Ret invoke(Args... arguments) {
return functor(arguments...);
}
};
// I know using macro is generally a bad idea but it's just shorter
#define wrap(f) wrap_known<decltype(f)>::invoke<f>
and then, when binding, use the macro like this:
luabind::def("Clear", wrap(glClear)),
luabind::def("Vertex4f", wrap(glVertex4f))
However, in your case, we have a bunch of member functions and not globals like above.
Here is the code for wrapping member functions with __stdcall calling convention:
template<typename Signature>
struct wrap_mem;
template<typename Sub, typename Ret, typename... Args>
struct wrap_mem<Ret(__stdcall Sub::*) (Args...)> {
template <Ret(__stdcall Sub::*functor) (Args...)>
static Ret invoke(Sub* subject, Args... arguments) {
return (subject->*functor)(arguments...);
}
};
#define wrap_member(f) wrap_mem<decltype(f)>::invoke<f>
Use it like this:
struct A {
int __stdcall my_method(double b) {
return 2;
}
};
// ...
luabind::class_<A>("A")
.def("my_method", wrap_member(&A::my_method))
Sometimes, however, you are not that lucky to know the function's address in compile time, and this happens with GLEW for example. For functions like glUniform*f, glGetUniformLocation, the "wrap" macro will not work, so I made another version for wrapping functions known at runtime:
template<typename Signature>
struct wrap_unknown;
template<typename Ret, typename... Args>
struct wrap_unknown<Ret (__stdcall*) (Args...)> {
template <Ret (__stdcall** functor)(Args...)>
static Ret invoke(Args... arguments) {
return (**functor)(arguments...);
}
};
#define wrap_ptr(f) wrap_unknown<decltype(f)>::invoke<&f>
(if above code scares you, it is actually a good sign)
Now you can bind GLEW functions like this:
luabind::def("Uniform4f", wrap_ptr(glUniform4f)),
luabind::def("GetUniformLocation", wrap_ptr(glGetUniformLocation))
Just don't ask me to write another version for binding pointers to members known at runtime :)
If you don't want to use C++11 for some reason, here you can find out how to pass function arguments and return value as template parameters in C++03.

How to convert out/ref extern parameters to F#

I've got a C# extern declaration that goes like this:
[DllImport("something.dll")]
public static extern ReturnCode GetParent(IntPtr inRef, out IntPtr outParentRef);
How to translate that to F#?
You can try something like the code below. I don't know what ReturnCode is, so the code below expects it is an integer. For any more complex type, you'll need to use [<Struct>] attribute as in the answer referenced by A-Dubb.
type ReturnCode = int
[<System.Runtime.InteropServices.DllImport("something.dll")>]
extern ReturnCode GetParent(System.IntPtr inRef, System.IntPtr& outParentRef);
To call the function, you'd write something like this:
let mutable v = nativeint 10
let n = GetParent(nativeint 0, &v)
BTW: Could you also post a sample C code that implements the function in something.dll? If yes, we could try running the solution before sending an answer...
Maybe this similar question will point you in the right direction. Looks like he used attributes at the parameter level for "in" and "out" F# syntax for P/Invoke signature using MarshalAs
For anyone else trying to use F# with EnvDte via PInvoke this may help:
[<System.Runtime.InteropServices.DllImport("ole32.dll")>]
extern unit CreateBindCtx(System.IntPtr inRef, IBindCtx& outParentRef);
[<System.Runtime.InteropServices.DllImport("ole32.dll")>]
extern unit GetRunningObjectTable(System.IntPtr inRef, IRunningObjectTable& outParentRef);
which apparently is slightly incorrect, but appears to work. the definition should be:
[<System.Runtime.InteropServices.DllImport("ole32.dll")>]
extern int CreateBindCtx(System.IntPtr inRef, IBindCtx& outParentRef);
[<System.Runtime.InteropServices.DllImport("ole32.dll")>]
extern int GetRunningObjectTable(System.IntPtr inRef, IRunningObjectTable& outParentRef);

Refactoring many identical DllImport declarations

I have a module which contains an interface to a native DLL; it looks like this:
// nvtt.dll binding module
module private NvTextureTools =
type NvttInputOptions = IntPtr
[<DllImport("nvtt", CallingConvention = CallingConvention.Cdecl)>]
extern NvttInputOptions nvttCreateInputOptions()
[<DllImport("nvtt", CallingConvention = CallingConvention.Cdecl)>]
extern void nvttDestroyInputOptions(NvttInputOptions)
[<DllImport("nvtt", CallingConvention = CallingConvention.Cdecl)>]
extern void nvttSetInputOptionsAlphaMode(NvttInputOptions, AlphaMode alphaMode)
[<DllImport("nvtt", CallingConvention = CallingConvention.Cdecl)>]
extern void nvttSetInputOptionsGamma(NvttInputOptions, float inputGamma, float outputGamma)
[<DllImport("nvtt", CallingConvention = CallingConvention.Cdecl)>]
extern void nvttSetInputOptionsWrapMode(NvttInputOptions, WrapMode mode)
(there are 5x more functions, but this should give the general idea).
Is there any way to specify the DllImport parameters just once? As far as I understand, I can't inherit from DllImport (it's sealed, and anyway I don't think it would work if it was not), and I can't use reflection to add the necessary attributes because I need them at compilation time.
I could make a brand new class with P/Invoke methods using reflection, but this will make calling them cumbersome.
Any thoughts?
I don't know about F#, but in C# you can do something like:
static const string DllName = "nvtt";
[DllImport(DllName, other params...)]
some function signature
[DllImport(DllName, other params...)=
some function signature
So that way the actual string is only declared once - the DllImport attributes themselves all still look a lot alike, but it makes changing things easier. I think you could do the same with CallingConvention, but I've never tried it with an enum.
Just in case you are using Visual Studio - it is possible to create a T4 template and generate all those nasty attributes. This is not an F# or VS specific solution however, any code generating tool would work.

P/Invoke Function Overloading

I'm attempting to P/Invoke a C library from F#, and have encountered a peculiar issue. I have a module containing all my extern functions. The underlying C library has two functions with the same name, but different arguments. This, of course, is not allowed in an F# module.
module C =
open System.Runtime.InteropServices
[<DllImport("libc", CallingConvention = CallingConvention.Cdecl)>]
extern int setValue(nativeint source, int value)
[<DllImport("libc", CallingConvention = CallingConvention.Cdecl)>]
extern int setValue(nativeint source, string value)
// the previous function declaration cause the following compile-time error:
// Duplicate definition of value 'setValue'
Is there some special way to work around this? I can not alter the C library.
The EntryPoint attribute should work (e.g. with an ordinal), if MSDN can be trusted (haven't tested in F#). Name your imported functions e.g. setValueInt() and setValueString().

Resources