Vadim's blog

Vadim
I am guessing that log4net wasn't included in order to simplify binary dependencies. But including it makes a lot of sense, log4net is quite popular in c# community.

Tracing in NMS

It easy to turn protocol tracing on in NMS client.
First of all you need to implement your trace class. I am lazy so I wrote an adapter for log4net.

/*
* ----------------------------------------------------------------------------
* "THE BEER-WARE LICENSE"
* Vadim Chekan wrote this file. As long as you retain this notice you
* can do whatever you want with this stuff. If we meet some day, and you think
* this stuff is worth it, you can buy me a beer in return
* ----------------------------------------------------------------------------
*/

using System;
using log4net;
using Apache.NMS;

namespace NMSTestConsole
{
public class TraceAdapter : Apache.NMS.ITrace
{
// keep logs on behalf of NMS Tracer
static readonly ILog _log = LogManager.GetLogger(typeof(Apache.NMS.Tracer));

public bool IsDebugEnabled {
get {return _log.IsDebugEnabled;}
}

public bool IsInfoEnabled {
get {return _log.IsInfoEnabled;}
}

public bool IsWarnEnabled {
get {return _log.IsWarnEnabled;}
}

public bool IsErrorEnabled {
get {return _log.IsErrorEnabled;}
}

public bool IsFatalEnabled {
get {return _log.IsFatalEnabled;}
}

public void Debug (string message)
{
_log.Debug(message);
}

public void Info (string message)
{
_log.Info(message);
}

public void Warn (string message)
{
_log.Warn(message);
}

public void Error (string message)
{
_log.Error(message);
}

public void Fatal (object message)
{
_log.Fatal(message);
}
}
}

Now you need hook it up somwhere at the very beginning of your program.

using Apache.NMS;
using Apache.NMS.ActiveMQ;
...
Tracer.Trace = new TraceAdapter();
log4net.Config.BasicConfigurator.Configure();

That's it. Run your program which communicates to ActiveMQ and see the result.

Getting more info

There is not too much information right now. Output looks like:

Parsing type: 1 with: Apache.NMS.ActiveMQ.OpenWire.V1.WireFormatInfoMarshaller
Parsing type: 2 with: Apache.NMS.ActiveMQ.OpenWire.V2.BrokerInfoMarshaller
Parsing type: 30 with: Apache.NMS.ActiveMQ.OpenWire.V2.ResponseMarshaller

Those numbers are ID of OpenWire packet types. Probably you would prefer something more human readable. You can achieve it by setting UseLogging property of TcpTransportFactory. The best wat of doing it is via broker url parameters:

IConnection conn = new ConnectionFactory("tcp://localhost:61616?transport.UseLogging=true").CreateConnection();

SENDING: WireFormatInfo[ Magic=System.Byte[] Version=2 MarshalledProperties={CacheEnabled=False, SizePrefixDisabled=False, StackTraceEnabled=False, TcpNoDelayEnabled=False, TightEncodingEnabled=False} ]
414 [-1225462896] DEBUG Apache.NMS.Tracer (null) - Parsing type: 1 with: Apache.NMS.ActiveMQ.OpenWire.V1.WireFormatInfoMarshaller
424 [-1225462896] INFO Apache.NMS.Tracer (null) - RECEIVED: WireFormatInfo[ Magic=System.Byte[] Version=3 MarshalledProperties={CacheEnabled=True, CacheSize=1024, SizePrefixDisabled=False, TightEncodingEnabled=True, MaxInactivityDuration=30000, MaxInactivityDurationInitalDelay=10000, StackTraceEnabled=True, TcpNoDelayEnabled=True} ]

SENDING: ConnectionInfo[ ConnectionId=ConnectionId[ Value=2957a0f0-a8b7-4abc-96ea-e5638b6d0baa ] ClientId=49b903c4-f55e-40b0-a78c-bde850b77ef8 Password= UserName= BrokerPath= BrokerMasterConnector=False Manageable=False ClientMaster=True ]
453 [-1225462896] DEBUG Apache.NMS.Tracer (null) - Parsing type: 2 with: Apache.NMS.ActiveMQ.OpenWire.V2.BrokerInfoMarshaller
463 [-1225462896] INFO Apache.NMS.Tracer (null) - RECEIVED: BrokerInfo[ BrokerId=BrokerId[ Value=ID:ubuntu-47413-1219891509239-0:0 ] BrokerURL=tcp://ubuntu:61616 PeerBrokerInfos=Apache.NMS.ActiveMQ.Commands.BrokerInfo[] BrokerName=localhost SlaveBroker=False MasterBroker=False FaultTolerantConfiguration=False DuplexConnection=False NetworkConnection=False ConnectionId=0 ]
463 [-1225462896] DEBUG Apache.NMS.Tracer (null) - Parsing type: 30 with: Apache.NMS.ActiveMQ.OpenWire.V2.ResponseMarshaller
463 [-1225462896] INFO Apache.NMS.Tracer (null) - RECEIVED: Response[ CorrelationId=1 ]
474 [-1211062496] INFO Apache.NMS.Tracer (null) - SENDING: SessionInfo[ SessionId=SessionId[ ConnectionId=2957a0f0-a8b7-4abc-96ea-e5638b6d0baa Value=1 ] ]
475 [-1225462896] DEBUG Apache.NMS.Tracer (null) - Parsing type: 30 with: Apache.NMS.ActiveMQ.OpenWire.V2.ResponseMarshaller
475 [-1225462896] INFO Apache.NMS.Tracer (null) - RECEIVED: Response[ CorrelationId=2 ]

Now you can have much better idea what your client is chatting about with the server.

NMS gotcha

I use NMS C# client to get messages from ActiveMQ and I faced a problem: despite there were messages in the queue my "consumer.ReceiveNoWait()" returned nothing.

After some code reading I realized the reason. My code looked like

using (IMessageConsumer consumer = session.CreateConsumer(queue, filter))
IMessage oldMessage;
while ((oldMessage = consumer.ReceiveNoWait() != null)
...

Apparently ReceiveNoWait does not send any request to the queue server. What it does is checking its own buffer of received messages.
When you create a Message Consumer then you notify the queue server that you want to receive messages from certain Target (Queue). But messages are not sent as registration response. So you have consumer registered but no messages yet.
Now, if you try to call consumer.ReceiveNoWait() right after registration you will get null. You need to wait a little bit before queue will send messages to the consumer.

/
Adding 100ms wait helps to address this problem:

using (IMessageConsumer consumer = session.CreateConsumer(queue, filter))
IMessage oldMessage;
while ((oldMessage = consumer.Receive(TimeSpan.FromMilliseconds(100))) != null)
...

Camel internals

I'm playing with Apache Camel project and boy, it is not easy to understand it internals!
Lets see how it is initialized.

SpringCamelContext(DefaultCamelContext).doStart() line: 543
SpringCamelContext.maybeDoStart() line: 165
SpringCamelContext.doStart() line: 160
SpringCamelContext(ServiceSupport).start() line: 47
SpringCamelContext.maybeStart() line: 95
SpringCamelContext.onApplicationEvent(ApplicationEvent) line: 114

So maybeStart calls start which calls doStart which calls maybeDoStart which calls doStart.
Now question from combinatorics: how many function names can we generate from 3 words: "maybe", "do", "start" :)))

XMPP and web

It seems XMPP protocol finally is getting recognition which it deserves. On 2008 OSCON there are some talks about using XMPP for async notifications.
Paper one. Paper two.

But like any new technology there are some confusion of how and when to use it.

So with XMPP my server will have to keep millions connections?

If we replace http pulling with a client listening to the server it means that all clients will have a persistent TCP connection to the server. If you have millions of clients then you are in trouble. Right?

No. XMPP address (JID or Jabber ID) looks like email. For example me@jabber.com or if you use google mail, then your JID is the same as your google email. Now lets say your server is mycompany.com. and your jabber server is at xmpp.mycompany.com. If a client from google chats with a service at your server then there is no direct TCP connection between client and your server. Instead client connects to Google, sends a message "to:room@xmpp.mycompany.com" and google will connect to xmpp.mycompany.com. Now, what will happen when 100 google clients will connect to a chat room at your server? Google server will re-use the same TCP connection to send your server a 100 "presence" xmpp packages. Handling even thousands packages is not a big deal at all.

So your server will handle as many connections as many unique domain your clients have. Which should be small enough number.

What about JMS?

XMPP does not replace JMS (Java Message Service) or any other messaging service. The principal difference is that JMS can guarantee delivery of a message or will return an error in case if message can not be delivered in time. With xmpp if client disconnected even for a short time then the client will not see any messages sent while he was offline. In other words XMPP is not for offline operations.

In JMS ( or any other queue architecture) sender can issue a message and as soon as it hit the queue there no need to be connected. Client can go offline and when it is back online pick up all message from the queue.

A good example for XMPP is a client application which shows properties of object. If other user in the system edited a property then server can issue a broadcast indicating ID of updated object. Every online application will refresh the property immediately. There is no need to listen to those messages if client is disconnected.

Now let's consider a replication system. Lets say we have a Corporation which is integrated with some Service Provider. Corporation uploaded its data to a Service Provider but now we have a dilemma: users can update data either in legacy Corporation application or in Service Provider interface.

Service Provider can create a queue for the Corporation and add a trigger in DB which dumps all changes to the queue. Corporation has an application which listens to the queue and updates legacy DB with changes made with Service Provider application.

Now imagine that Corporation has been disconnected from Service Provider for some time (maintenance, networking problems, etc). Obviously when it is back online it is crucial to receive all messages issues during offline period. XMPP is no way the tool for this task. JMS is.

XMPP does not provide reliability. (Correction: there exist a publish-subscribe extension for XMPP which has persistent messages option. But not all servers have it implemented so for queuing purposes it is still better to use a reputable queue system).

When pull is better

XMPP helps in situation when updates happen not too often. If message traffic is high then each time you perform pull you are quite sure that you will get data. So there is not too much sense in implementing asynchronous notification. Unless you have strong requirements to the reaction time.

What about WS-notification/SOAP

The big problem is that HTTP protocol is one-way communication. Basically server never talks to the client. Server even does not know how to reach its clients. WS-Notification attempts to solve this problem by client calling the server and telling the server where it can find the client if event happen. In case of corporate integration it may work. You assault your IT department and get a server where you put your little callback web service. But there exist cases which never will work. Web browser generally speaking can not be accessible because of firewalls. So WS-Notification is not a generic solution. XMPP is a solution because it initiates connection to the server and remains connected so server always can send an even to the client.

OpenID Provider Identifier vs Claimed Identifier

It is not emphasized in OpenID specification (version 2.0) but when you log in into OpenID enabled site, you can either enter your Claimed Identifier, for example mylogin.pip.verisignlabs.com or alternatively you can enter just your provider address, for example "yahoo.com" and site will figure it out that the provider you've entered supports OpenID.

How it does this? Simlpe.
According to the spec url you entered is normalized: added "http://" and followed the redirect links. If you enter "yahoo.com" in your browser, you will be redirected to "http://yahoo.com/". Now, lets see how OpenID server location is discovered with help of small Ruby script.

require 'net/http'
h = Net::HTTP.new('www.yahoo.com')
resp = h.head('/', 'Accept' => 'application/xrds+xml')
puts "Code = #{resp.code}"
resp.each_key {|key| puts "#{key} = #{resp[key]}"}

This script issues "HEAD" http command to the yahoo server. Note "application/xrds+xml" Accept header. If the server supports OpenID it should return a header indicating where OpenID server can be found.
The result is:

vadim@ubuntu:~/Projects/yadis_test$ ruby test.rb
last-modified = Sat, 15 Mar 2008 06:22:40 GMT
cache-control = private
vary = User-Agent
connection = close
p3p = policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM ..."
content-type = text/html; charset=utf-8
date = Sat, 15 Mar 2008 07:01:22 GMT
content-length = 9533
x-xrds-location = http://open.login.yahooapis.com/openid20/www.yahoo.com/xrds
accept-ranges = bytes
vadim@ubuntu:~/Projects/yadis_test$

Here we go. Yahoo OpenID server is located at http://open.login.yahooapis.com/openid20/www.yahoo.com/xrds
If you are curious enough, you can go to this url and download so-called "yadis" file, which is xml declaration of supported services and options.

Funny.
From RFC-2631: "In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is explicitly specified" :)