On to to to-dos

Back from the grave.

I moved EVERYTHING from Windows XP 32 to Windows 7 64 bit. This has proven to be such a black hole that... well, everything fell into it.

It's weird that that I cannot find a guide on porting an application. There are so many deprecated Win32 functions that need to be replaced with x64 counterparts. But a quick google does not find a tutorial on what to do, and what to avoid.

Well, we will see.

Ant Task to generate GWT Classes and servlets

Yet Another Ant Task.
I made a task which given:
A source dir, a source classpath, and destination dir,
A series of source interfaces and a new GWT name,

My task will generate all the GWT plumbing. Namely the Service interface, the ServiceAsync interface, and a Servlet Implementation. If you specify a delegate, the the servlet will implement the interface by calling the methods in the delegate.
Pretty sweet.

I learned a lot about how "Generic" info about classes can be obtained through the reflection API. You might think that all the type parameter stuff is erased, a-la
Section 4.6 of the Java spec., but it is all there in the class, available through reflection. On the other hand, this leads to a paradox, because the it's not supposed to be there.

For example, let's say you have:

public interface Foo{
List<String> someMethod();
}

The compiler halts with an error if you try to implement Foo with this class:

public class Bar implements Foo{
List<String> someMethod(){return new ArrayList<String>();}
}

Because the type parameter of the returned List<String> has been "erased", and the return type in the interface is actually just List.

But if you were using reflection to generate class source you would still find the <String> type parameter. So to get your implementation to compile, you must discard the Type info.

At least that's the way it seems to work :)

How to make JNI apps reloadable

JNI has this annoying "feature" , that makes is very difficult to run an application ( or ant task, or whatever) that loads a native library.
When you load a class ( lets call it Foo ) that loads loads a DLL, loadLibrary is called on the ClassLoader that loaded Foo.
If you try to load the Foo class again, and the library is still loaded, then loadLibrary will fail with a java.lang.UnsatisfiedLinkError.

The DLL that Foo loaded will NOT be unloaded when Foo is garbage collected. The DLL will not be unloaded until the ClassLoader, and all the other classes that were loaded with Foo's ClassLoader are garbage collected. If that ClassLoader is the System ClassLoader, it will never be unloaded while the JVM is running.

This makes it an exceptional pain to try and re-deploy an app to an application server.
There is a lot of whining about this, and a lot of bad advice. So here is a WORKING solution.

A) All the classes that load native libraries (JNI clients) should implement interfaces for their use.
B) All the other classes should ONLY reference the interfaces, never the classes themselves.
C) JNI client classes must be loaded by a separate classloader then the other classes.
D) The instances of the JNI client classes must be instantiated by reflection, not by directly calling a class’s constructor. This prevents the class being loaded by a non-JNI ClassLoader

To accomplish this, I created a VolitileFactory class, that subclasses URLClassLoader. It has a makeInstance method, which loads the JNI client classes, and invokes the correct constructor via reflection. The VolitileFactory instance is a field of the Anchor class. Anchor has static methods to obtain implementers of the JNI client’s interfaces. Anchor holds the only reference to the factory. When the app shuts down, that reference is deleted, System.gc() is called, and the libraries unload.

Summer break is over, back to the Toy.

I re-implemented the event loop and the event objects to be in Java instead of native code.

There is a Java EventMachine. A client uses getTheEventMachine to get only instance. It maintains a timer and a FIFO blocking queue of SimpleEvents. SimpleEvents are added to the queue by the addEvent method of EventMachine. At the preset interval, the queue is locked, and the SimpleEvents in it are dispatched to processEvent in the UpperDeck one at a time, until they are all processed, or until a set amount of time has passed. Then, dispatching ceases and the queue is unlocked. processEvent in UpperDeck processes the event, and in a finally clause, calls the delete method of the event.

This system is used by native code via a C++ JEventMachineConnector class. This maintains a global reference to a thread in the EventMachine where addEvent executes. So when addEvent is called on JEventMachineConnector, it joins the Java thread, calls addEvent on EventMachine, and upon completion, detaches from the thread.

The SimpleEvent interface is an immutable collection of three properties: source, type and message, and a delete method. However the implementation of SimpleEvent is not simple. An implementation consists of a Java object SimpleEventJavaImpl, and it’s native shadow. The Java object implements SimpleEvent, exports a delete method, and maintains a pointer to the native shadow object. This delete method is actually implemented natively. Upon garbage collection, if the native resources have not been freed, SimpleEventJavaImpl calls its own delete method. The native shadow object maintains a link to the Java object, and native versions of the initial value of the properties of the SimpleEvent. It also implements logic to create SimpleEventJavaImpl objects from the native side. The hairiness of SimpleEvent implementation comes from the logic to instantiate one from the native or Java side, and then to delete all native resources.

The Toy also grew much hair implementing Java<->Native exception handling, and other error management. I learned far more then I wanted to know about C++ templates, Unicode on Windows, and why the C++ preprocessor is pure freaking EVIL.

Passing wrapped C++ objects to Java

In the Toy, C++ code generates an Event, and passes it to Java for processing.
The flow is
Client class (Cclient) creates C++ SimpleEvent (launchEvent)
cClient calls getJEMConnector()->addEvent(*launchEvent);
addevent attaches the C++ thread to the JVM,
creates a Java SimpleEvent wrapper (jSimpleEvent)
calls addEvent(jSimpleEvent) on the Java EventMachine

An now it's up to Java to further handle the event.

Object life cycles are tricky when there are multiple threads. Calling a Java constructor from C++ is no fun because of the weird signature stuff, i.e. "", "(JZ)V" to call the constructor that takes a (long)pointer and a boolean argument.
Then there is Local/Global reference issue. DeleteGlobalRef must be called within C++, before the object is destroyed, on some JavaENV, that should have been cached earlier. This is on top of the memory management problem, as discussed earlier.

I'm going to do some memory leak checks now. I'll be surprised if there are none, because nowhere in C++ do I call DeleteGlobalRef for the SimpleEvent wrapper.

Allocate objects from the JVM

My Event and threading mechanics are all going to be in Java. But I still create some events from C++, wrap them in a Java class, and add them to the queue. When they are processed, I destroy them, and clear the queue, so there are no references.

This created a memory management problem. The "new" was in c++, but the "delete" is in Java. So I ended up overriding the new and delete operators, and having them use the JVM for resources. This seems to work.

Note to self, varargs stuff are all macros

So, I leared how to use varargs.

Not just declaring a varargs function and getting the arguments.
No, the trick was to get a va_list and pass it on to something like vswprintf. My problem was that whenever I had a number in my va_list, it would get scrambled in the output.

The thing to learn was that all the varargs stuff is just pointer arithmetic on a single char *, obfuscated by macros. The macros create hidden variables, which change state on every use of va_start(), va_arg(), va_list(). So, you can't use any of the arguments passed to your function if you want to pass that va_list on to another function. If you ever call va_arg(), you must reset everything bay calling va_end(), va_start() before you pass the va_list.

It's pretty rotten that
A) None of this stuff follows macro naming conventions, i.e VA_START
B) The side effects of using va_start and va_args are undocumented, not even on faqs or forums.