Monday 3 October 2011

Java Native Access (JNA)

What is JNA?

JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required. This functionality is comparable to Windows' Platform/Invoke and Python's ctypes. Access is dynamic at runtime without code generation.
JNA allows you to call directly into native functions using natural Java method invocation. The Java call looks just like it does in native code. Most calls require no special handling or configuration; no boilerplate or generated code is required.
The JNA library uses a small native library stub to dynamically invoke native code. The developer uses a Java interface to describe functions and structures in the target native library. This makes it quite easy to take advantage of native platform features without incurring the high overhead of configuring and building JNI code for multiple platforms.
While some attention is paid to performance, correctness and ease of use take priority.
JNA includes a platform library with many native functions already mapped as well as a set of utility interfaces that simplify native access.

Features

  • Automatic mapping from Java to native functions, with simple mappings for all primitive data types
  • Runs on most platforms which support Java
  • Automatic conversion between C and Java strings, with customizable encoding/decoding
  • Structure and Union arguments/return values, by reference and by value
  • Function Pointers, (callbacks from native code to Java) as arguments and/or members of a struct
  • Auto-generated Java proxies for native function pointers
  • By-reference (pointer-to-type) arguments
  • Java array and NIO Buffer arguments (primitive types and pointers) as pointer-to-buffer
  • Nested structures and arrays
  • Wide (wchar_t-based) strings
  • Native long support (32- or 64-bit as appropriate)
  • Demo applications
  • Supported on 1.4 or later JVMs (earlier VMs may work with stubbed NIO support)
  • Customizable marshalling/unmarshalling (argument and return value conversions)
  • Customizable mapping from Java method to native function name, and customizable invocation to simulate C preprocessor function macros
  • Support for automatic Windows ASCII/UNICODE function mappings
  • Varargs support
  • Type-safety for native pointers
  • VM crash protection (optional)
  • Optimized direct mapping for high-performance applications. 

How To Get Started Using JNA

Java Native Access (JNA) has a single component, jna.jar; the supporting native library (jnidispatch) is included in the jar file. JNA is capable of extracting and loading the native library on its own, so you don't need additional configuration. JNA falls back to extraction if the native library is not already installed on the local system somewhere accessible to System.loadLibrary (see information on library loading). The native library is also available in platform-specific jar files for use with Java Web Start.
  1. Download jna.jar from the download page.
  2. Compile and run this short example, which maps the printf function from the standard C library and calls it. Be sure to include jna.jar in the classpath:
    
    package com.sun.jna.examples;
    
    import com.sun.jna.Library;
    import com.sun.jna.Native;
    import com.sun.jna.Platform;
    
    /** Simple example of JNA interface mapping and usage. */
    public class HelloWorld {
    
        // This is the standard, stable way of mapping, which supports extensive
        // customization and mapping of Java to native types.
        public interface CLibrary extends Library {
            CLibrary INSTANCE = (CLibrary)
                Native.loadLibrary((Platform.isWindows() ? "msvcrt" : "c"),
                                   CLibrary.class);
        
            void printf(String format, Object... args);
        }
    
        public static void main(String[] args) {
            CLibrary.INSTANCE.printf("Hello, World\n");
            for (int i=0;i < args.length;i++) {
                CLibrary.INSTANCE.printf("Argument %d: %s\n", i, args[i]);
            }
        }
    }
    
  3. Identify a native target library that you want to use. This can be any shared library with exported functions. Many examples of mappings for common system libraries may be found in the platform package.
  4. Make your target library available to your Java program. There are two ways to do this:
    • The preferred method is to set the jna.library.path system property to the path to your target library. This property is similar to java.library.path but only applies to libraries loaded by JNA.
    • Change the appropriate library access environment variable before launching the VM. This is PATH on Windows, LD_LIBRARY_PATH on Linux, and DYLD_LIBRARY_PATH on OSX.
  5. Declare a Java interface to hold the native library methods by extending the Library interface. Following is an example of mapping for the Windows kernel32 library.
    
    package com.sun.jna.examples.win32;
    
    import com.sun.jna.*;
    
    // kernel32.dll uses the __stdcall calling convention (check the function
    // declaration for "WINAPI" or "PASCAL"), so extend StdCallLibrary
    // Most C libraries will just extend com.sun.jna.Library,
    public interface Kernel32 extends StdCallLibrary { 
        // Method declarations, constant and structure definitions go here
    }
  6. Within this interface, define an instance of the native library using the Native.loadLibrary(Class) method, providing the native library interface you defined in step (5).
    
        Kernel32 INSTANCE = (Kernel32)
            Native.loadLibrary("kernel32", Kernel32.class);
        // Optional: wraps every call to the native library in a
        // synchronized block, limiting native calls to one at a time
        Kernel32 SYNC_INSTANCE = (Kernel32)
            Native.synchronizedLibrary(INSTANCE);
    

    The INSTANCE variable is for convenient reuse of a single instance of the library. Alternatively, you can load the library into a local variable so that it will be available for garbage collection when it goes out of scope. A Map of options may be provided as the third argument to loadLibrary to customize the library behavior; some of these options are explained in more detail below. The SYNC_INSTANCE is also optional; use it if you need to ensure that your native library has only one call to it at a time.
  7. Declare methods that mirror the functions in the target library by defining Java methods with the same name and argument types as the native function (refer to the basic mappings below or the detailed table of type mappings). You may also need to declare native structures to pass to your native functions. To do this, create a class within the interface definition that extends Structure and add public fields (which may include arrays or nested structures).
    
    public static class SYSTEMTIME extends Structure {
        public short wYear;
        public short wMonth;
        public short wDayOfWeek;
        public short wDay;
        public short wHour;
        public short wMinute;
        public short wSecond;
        public short wMilliseconds;
    }
    
    void GetSystemTime(SYSTEMTIME result);
        
  8. You can now invoke methods on the library instance just like any other Java class. For a more extensive example, see the WindowUtils and ShapedWindowDemo classes.
    
    Kernel32 lib = Kernel32.INSTANCE;
    SYSTEMTIME time = new SYSTEMTIME();
    lib.GetSystemTime(time);
    System.out.println("Today's integer value is " + time.wDay);
    
  9. Alternatively, you may declare a class to hold your native methods, declare any number of methods with the "native" qualifier, and invoke Native.register(String) in the class static initializer with your library's name. See JNA direct mapping for an example.
If the C header files for your library are available, you can auto-generate a library mapping by using Olivier Chafik's excellent JNAerator utility. This is especially useful if your library uses long or complicated structures where translating by hand can be error-prone. See the JavaDoc overview for more detailed information about JNA usage.

Default Type Mappings

Java primitive types (and their object equivalents) map directly to the native C type of the same size.
Native Type Size Java Type Common Windows Types
char 8-bit integer byte BYTE, TCHAR
short 16-bit integer short WORD
wchar_t 16/32-bit character char TCHAR
int 32-bit integer int DWORD
int boolean value boolean BOOL
long 32/64-bit integer NativeLong LONG
long long 64-bit integer long __int64
float 32-bit FP float
double 64-bit FP double
char* C string String LPTCSTR
void* pointer Pointer LPVOID, HANDLE, LPXXX
Unsigned types use the same mappings as signed types. C enums are usually interchangeable with "int". A more comprehensive list of mappings may be found here

Using Pointers and Arrays

Primitive array arguments (including structs) are represented by their corresponding Java types. For example:

// Original C declarations
void fill_buffer(int *buf, int len);
void fill_buffer(int buf[], int len); // same thing with array syntax

// Equivalent JNA mapping
void fill_buffer(int[] buf, int len);
NOTE: if the parameter is to be used by the native function outside the scope of the function call, you must use Memory or an NIO Buffer. The memory provided by a Java primitive array will only be valid for use by the native code for the duration of the function call.
Arrays of C strings (the char* argv[] to the C main, for example), may be represented by String[] in Java code. JNA will pass an equivalent array with a NULL final element.

Using Structures/Unions

When a function requires a pointer to a struct, a Java Structure should be used. If the struct is passed or returned by value, you need only make minor modifications to the parameter or return type class declaration. Typically you define a public static class derived from Structure within your library interface definition. This allows the structure to share any options (like custom type mapping) defined for the library interface.
If a function requires an array of struct (allocated contiguously in memory), a Java Structure[] may be used. When passing in an array of Structure, it is not necessary to initialize the array elements (the function call will allocate, zero memory, and assign the elements for you). If you do need to initialize the array, you should use the Structure.toArray method to obtain an array of Structure elements contiguous in memory, which you can then initialize as needed.
Unions are generally interchangeable with Structures, but require that you indicate which union field is active with the setType method before it can be properly passed to a function call.

Using By-reference Arguments

When a function accepts a pointer-to-type argument you can use one of the ByReference types to capture the returned value, or subclass your own. For example:

// Original C declaration
void allocate_buffer(char **bufp, int* lenp);

// Equivalent JNA mapping
void allocate_buffer(PointerByReference bufp, IntByReference lenp);

// Usage
PointerByReference pref = new PointerByReference();
IntByReference iref = new IntByReference();
lib.allocate_buffer(pref, iref);
Pointer p = pref.getValue();
byte[] buffer = p.getByteArray(0, iref.getValue());
    
Alternatively, you could use a Java array with a single element of the desired type, but the ByReference convention better conveys the intent of the code. The Pointer class provides a number of accessor methods in addition to getByteArray() which effectively function as a typecast onto the memory.
Type-safe pointers may be declared by deriving from the PointerType class.

Customized Mapping from Java to Native (Types and Function Names)

The TypeMapper class and related interfaces provide for converting any Java type used as an argument, return value, or structure member to be converted to or from a native type. The example w32 API interfaces use a type mapper to convert Java boolean into the w32 BOOL type. A TypeMapper instance is passed as the value for the TYPE_MAPPER key in the options map passed to Native.loadLibrary. Alternatively, user-defined types may implement the NativeMapped interface, which determines conversion to and from native types on a class-by-class basis.
You may also customize the mapping of Java method names to the corresponding native function name. The StdCallFunctionMapper is one implementation which automatically generates stdcall-decorated function names from a Java interface method signature. The mapper should be passed as the value for the OPTION_FUNCTION_MAPPER key in the options map passed to the Native.loadLibrary call.
Refer to this table in the overview for a complete list of built-in type mappings.

Callbacks/Closures (Function Pointers)

Callback declarations consist of a simple interface that extends the Callback interface and implements a callback method (or defines a single method of arbitrary name). Callbacks are implemented by wrapping a Java object method in a little bit of C glue code. The simplest usage resembles using anonymous inner classes to register event listeners. Following is an example of callback usage:

// Original C declarations
typedef void (*sig_t)(int);
sig_t signal(sig_t);

// Equivalent JNA mappings
public interface CLibrary extends Library {
    int SIGUSR1 = 30;
    interface sig_t extends Callback {
        void invoke(int signal);        
    }
    sig_t signal(int sig, sig_t fn);
    int raise(int sig);
}
...
CLibrary lib = (CLibrary)Native.loadLibrary("c", CLibrary.class);
// WARNING: you must keep a reference to the callback object
// until you deregister the callback; if the callback object
// is garbage-collected, the native callback invocation will
// probably crash.
CLibrary.sig_t fn = new CLibrary.sig_t() {
    public void invoke(int sig) {
        System.out.println("signal " + sig + " was raised");
    }
};
CLibrary.sig_t old_handler = lib.signal(CLibrary.SIGUSR1, fn);
lib.raise(CLibrary.SIGUSR1);
...
Here is a more involved example, using the w32 APIs to enumerate all native windows:

// Original C declarations
typedef int (__stdcall *WNDENUMPROC)(void*,void*);
int __stdcall EnumWindows(WNDENUMPROC,void*);

// Equivalent JNA mappings
public interface User32 extends StdCallLibrary {
    interface WNDENUMPROC extends StdCallCallback {
        /** Return whether to continue enumeration. */
        boolean callback(Pointer hWnd, Pointer arg);
    }
    boolean EnumWindows(WNDENUMPROC lpEnumFunc, Pointer arg);
}
...
User32 user32 = User32.INSTANCE;

user32.EnumWindows(new WNDENUMPROC() {
    int count;
    public boolean callback(Pointer hWnd, Pointer userData) {
        System.out.println("Found window " + hWnd + ", total " + ++count);
        return true;
    }
}, null);
If your callback needs to live beyond the method invocation where it is used, make sure you keep a reference to it or the native code will call back to an empty stub after the callback object is garbage collected. Proxy wrappers are automatically generated for function pointers found within structs initialized by native code. This facilitates calling those functions from Java.

Invocation from Dynamically-Typed Languages

Languages such as Jython or JRuby may find it more convenient to access the NativeLibrary and Function classes directly rather than establishing a dedicated interface. Here's a brief example of using JNA from JRuby:

require 'java'

module Libc
  @@lib = com.sun.jna.NativeLibrary.getInstance("c")
  @@ptr_funcs = [ 'fopen', 'malloc', 'calloc' ]
  def self.method_missing(meth, *args)
    if @@ptr_funcs.include?(meth.to_s)
      @@lib.getFunction(meth.to_s).invokePointer(args.to_java)
    else
      @@lib.getFunction(meth.to_s).invokeInt(args.to_java)
    end
  end
  O_RDONLY = 0
  O_WRONLY = 1
  O_RDWR = 2
end

Libc.puts("puts from libc")
Libc.printf("Hello %s, from printf\n", "World")

file = Libc.open("/dev/stdout", 1, Libc::O_WRONLY)
n = Libc.write(file, "Test\n", 5)
puts "Wrote #{n} bytes via Libc"

path = "/dev/stdout"
fp = Libc.fopen(path, "w+")
Libc.fprintf(fp, "fprintf to %s via stdio\n", path)
Libc.fflush(fp)
Libc.fclose(fp)

Platform Library

JNA includes platform.jar that has cross-platform mappings and mappings for a number of commonly used platform functions, including a large number of Win32 mappings as well as a set of utility classes that simplify native access. The code is tested and the utility interfaces ensure that native memory management is taken care of correctly.
Before you map your own functions, check the platform package documentation for an already mapped one.
Platform-specific structures are mapped by header. For example, ShlObj.h structures can be found in com.sun.jna.platform.win32.ShlObj. Platform functions are mapped by library. For example, Advapi32.dll functions can be found in com.sun.jna.platform.win32.Advapi32. Simplified interfaces (wrappers) for Advapi32.dll functions can be found in com.sun.jna.platform.win32.Advapi32Util.
Cross-platform functions and structures are implemented in com.sun.jna.platform. These currently include the following.
  • FileMonitor: a cross-platform file system watcher
  • FileUtils: a cross-platform set of file-related functions, such as move to the recycle bin
  • KeyboardUtils: a cross-platform set of keyboard functions, such as finding out whether a key is pressed
  • WindowUtils: a cross-platform set of window functions, providing non-rectangular shaped and transparent windows

No comments:

Post a Comment