upgrading JBother's file transfer - Smack 2.2.0

After I have finished implementing my own version of (direct) file transfer, I decided to upgrade JBother to the latest Smack library (I have used a bleeding-edge nightly build from March, 12) because it includes file transfer already. This is going to be a nice training before embarking on newest JChat project - creating our own version of Jabber communicator :-). Well, I must say that new code works ok, it doesn't implement 'direct' SOCKS5 file transfer - it uses proxified but the proyx is running on the same host as Initiator - which makes it little bit more universal and doesn't bring too much of an overhead (the difference is one extra TCP connection on Initiator's side, from his data stream socket to the 'bogus' proxy's).

So, at the end of the day I was able to transfer files between JBother and PSI using SOCKS5 method - not too bad! Still to come: find a way to specify different proxy65 hosts than local host (to actually see the file transfer from behind the NAT working) and get 'I would like to use the older, deprecated IBB over SOCKS5 method' option.

I have found some inconsistencies in current Smack code that has been picked up by (always accurate, picky PSI):

GOOD:

<iq from=\"admin@devpc4.ccb.nc3a.nato.int/JBother\" type=\"set\" to=\"test@devpc4.ccb.nc3a.nato.int/Psi\" id=\"T5qLw-7\" >
<si xmlns=\"http://jabber.org/protocol/si\" profile=\"http://jabber.org/protocol/si/profile/file-transfer\" id=\"jsi_4019840249536931701\" >
<file xmlns=\"http://jabber.org/protocol/si/profile/file-transfer\" size=\"27976321\" name=\"xen-3.0.1-install-x86_32p.tgz\" >
<desc>asdfasdf</desc>
</file>
<feature xmlns=\"http://jabber.org/protocol/feature-neg\">
<x xmlns=\"jabber:x:data\" type=\"form\" >
<field type=\"list-single\" var=\"stream-method\" >
<option>
<value>http://jabber.org/protocol/bytestreams</value>
</option>
<option>
<value>http://jabber.org/protocol/ibb</value>
</option>
</field>
</x>
</feature>
</si>
</iq>

BAD:

<iq from=\"admin@devpc4.ccb.nc3a.nato.int/JBother\" type=\"set\" to=\"test@devpc4.ccb.nc3a.nato.int/Psi\" id=\"T5qLw-7\" >
<si xmlns=\"http://jabber.org/protocol/si\" profile=\"http://jabber.org/protocol/si/profile/file-transfer\" id=\"jsi_4019840249536931701\" >
<file xmlns=\"http://jabber.org/protocol/si/profile/file-transfer\" size=\"27976321\" name=\"xen-3.0.1-install-x86_32p.tgz\" >
<desc>asdfasdf</desc>
</file>
<feature xmlns=\"http://jabber.org/protocol/feature-neg\">
<x xmlns=\"jabber:x:data\" type=\"form\" >
<field type=\"list-multi\" var=\"stream-method\" >
<option>
<value>http://jabber.org/protocol/bytestreams</value>
</option>
<option>
<value>http://jabber.org/protocol/ibb</value>
</option>
</field>
</x>
</feature>
</si>
</iq>

I have notified Alex Wenckus, who maintains Smack's filetransfer code.

JBother 'direct' file transfer - more testing

I added a simple class 'Socks5AuthenticationMessage' to JBother's file transfer in order to handle SOCKS5 authentication messages more easily. I have also started testing of new old fuctionality across subnets and OSes; it proved that it worked as long as the host/port properties are OK... so instead of trusting the user to put the correct values, JBo is trying to fill them out as much as possible. Hopefully this will result in less complaints from users :-).

Also (re)started working on Wildfire plugin for logging Jabber messages. After SIMEX lots of people reported that it would be nice to be able to view conversations... this would be something like Bandersnatch but better - with the possibility to view our extensions to XMPP.

some JBother news

It's been avery long time since anything interesting was put here so I guess it's good moment to write down what was going on during the last couple of weeks. We have successfuly completed SIMEX '06, where Jabber service was performing cery well - I'm very happy with new Wildfire server. As for JBother, it was working very good for most of the time; plugins worked OK although our favourite bug hunter Reiner managed to find some. Main thing that was not all right was (infamous) file transfer. That's why I decided to work on it now.

File transfer is implemented in latest version of JBother to use 'proxy65' method to make transfers over the internet possible. What we need, however, is 'direct' filetransfer, as described in JEP-0065. That was why, among other things, we were using older (0.8.8 + some corrections) version of JBother. Rigth now we will switch to the latest (0.8.9) version of JBo and I'm going to put my changes to make the 'direct' SOCKS5 filetransfer possible. I have pre-tested it today with JBother and PSI and it worked; it also worked one-way with Tkabber, due to the bug in Tkabber (it can't handle mor than one option offered in stream initiation; and JBother offers now both StreamHost-based and IBB (in-band-bytestreams, older method).

There is also more stuff coming up when it comes to Jabber; we are considering using Jabber with working filetransfer (hopefully it will come very soon! :-) to ingest files into CSD, instead of FTP based solution we are using right now; also we would like to separate JChat client from FAST and create a standalone IM/Conference client. This is still in alpha-phase right now but sound quite interesting.

GenJar update; new releases of JBother & plugins

It's been long time since last writeup in this area but this of course doesn't mean that I was doing nothing! :-). First of all: I got GenJar utility working with SALT plugin (not with SSR), but after the change of schema used for SALTs the gain in size of the plugin is not big - ~200 kB (well, it's still something!). I couldn't get SSR working but I think it is also feasible - I just didn't spend too much time on this - and also worth doing, right now even after switching to new, more space saving schema - the plugin is still ~3 MB.

OK, lots of thing changed: as I have already mentioned, we are using new version of schema for SSR and SALT messages. It is more efficient and compact and has less dependencies on other schemas, notably GMTI (the size od XML Beans-generated code was cut by ~1 MB). Thanks to XML Beans approach, adapting to new schema was quite easy.

Other major improvement (and direct legacy of TIE'05) was introducing 'service discovery' mechanism to both plugins. The main objective of it is to ensure that the other end, the Target Jabber user can understand what is being sent to it. After TIE'05, the 'Crash Team' (sic!) started a long e-mail conversation what approach would be the best and decided that 'annotated presence' is that best. Annotated presence is a way of advertising certain properties of Jabber resource by embedding information about it in tag, similar as it is done in JEP-0115: Entity Capabilities. It turned out that Smack library is pretty much already prepared to handle package extensions (elements like ). In order to use it I had to make changes to core JBother code but I managed to contact and consult the JBother main developer beforehand and got 'permission' to do it (thanks again, Adam!).
I have also added other proposed service discovery mechanism based on standard JEP-0030: Service Discovery mechanism (just in case that we will decide to switch to it instead). Settings for controlling the service discovery in JBother plugins are enclosed in file 'serviceDiscovery.property'.
Other improvement asked for during and after the TIE was a 'history' window, with list of all send and received SSR (after some time, users were loosing track on what they did).

So, what comes next? In the new release (I hope I will make it before SIMEX), I'd like to look closely on two plugins and eliminate the double code that definitely is there. Some more GUI improvements would also be nice to have :-).

GenJar JAR utility

In neverending pursuit to make our code faster, more eye-candy and, well, less bloated, I stumbled upon utility called GenJar. It is supposed to help build JAR files that contain only the needed classes - what it does is scans the class directory for root class(es) (provided in config file) and builds JAR with other classes that are needed by those root class to work. Very nifty, I was able to reduce the size of GMTI Generator's distribution JAR by 30% but it is not working good when it comes to SSR and SALT plugins, at least not with my configuration (I need to figure out what exactly is going on when plugin's classes are loaded).

GMTI Generator: better rendering in UI

I've been working on CGMTI Generator, to be more precise on UI part (sleazy JTree rendering, more details in this writeup). Now it's working much, much better: the whole data model for the tree is now a little more complicated (the lazy solution, converting to XML and displaying the elements as tree leaves turned out to be more costly when it came to editing - and displaying, especially large files).

Right now the model is returning regular GMTI package components (package header, all the segments, target reports). Before I had problems with slow rendering of those objects, but it turned out that I wanted to display too much. Right now I'm displaying only the most important information, and it's fast.

I plan to do some more last minute touches and post the new version (0.4) on the portal.

GMTI Generator: packet splitting almost done

Today I was working on packet splitting in GMTI Generator and (ta-ta!) it is working! Right now it assumes that dwell segment comes as last in the GMTI packet, but I guess this is the standard way, approved by the community earlier. So far it hasn't been tested too much, I'm going to do it tomorrow.

The other thing that need to be done is re-designing the way the GMTI packets are displayed and handled. Right now, I'm using simple solution: I'm taking the GMTI packet, convert is to XML (the method was already there) and then display it in the JTree. The way as it is right now requires lots of object->XML conversions and is not very robust and memory efficient; I have put it in there only because I was too lazy to write a proper tree renderer :-). This needs to be changed right now.