Need help understanding serializer

[Christian Kreibich]

Hi Robin,


I've been looking at the serialization code in Bro for a while now and
I've hit a dead end. It'd be really cool if you could help me out
because I think I simply don't get it :)

I think I've basically understood the concepts of SerialObj, Serializer
and SerializationFormat -- I think I would have structured things the
same way. I'm getting lost in the details though:

- When exactly and why does a class have to implement Unserialize() and
Serialize()? What's their relationship to DoSerialize() and
DoUnserialize()? The comments in SerialObj.h are a bit vague in that
regard.

- It seems the idea of sending ahead an identifier to the receiving end
that tells it what to do with the following data exists three times:

      *  in the SER_xxx constants and the factory approach in
        IMPLEMENT_SERIAL
      *  in the character constants 'i', 'e', 's' etc in Serializer.cc
      *  in the MSG_xxx constants in RemoteSerializer.cc.

I think the latter are partially internal to the remote<->local
communication and can hence mostly be ignored for understanding the
serialization code, right? If you could quickly explain the difference
between the first two that'd be great.

The next thing I noticed are the hardcoded (un)seralization methods in
Serializer: 

        bool Serialize(const ID& id, const SerialInfo& info);
        bool Serialize(const char* func, val_list* args);
        bool Serialize(const StateAccess& s);
        bool Serialize(const Connection& c);
        bool Serialize(const Timer& t);

        virtual void GotID(ID* id, Val* val) = 0;
        virtual void GotEvent(const char* name, double time,
                                EventHandlerPtr event, val_list* a) = 0;
        virtual void GotFunctionCall(const char* name, double time,
                                Func* func, val_list* args) = 0;
        virtual void GotStateAccess(StateAccess* s) = 0;
        virtual void GotTimer(Timer* t) = 0;
        virtual void GotConnection(Connection* c) = 0;

        bool UnserializeID();
        bool UnserializeCall();
        bool UnserializeStateAccess();
        bool UnserializeTimer();
        bool UnserializeConnection();

Are these special in a way to have them implemented this way? Couldn't
there be a "received" callback per SER_xxx constant that resides as
a static method in the serializable classes themselves? So we can avoid
hardcoding anything?

- Following the comments in SerialObj.h, I see what I need to do to make
a class's objects serializable. I presume that the correct way to ship
an object to a serializer is by calling SerialObj::Serialize() with the
appropriate serializer. What are my options for picking them up at the
receiving end?

Oh and RemoteSerializer::ProcessSerialization() calls Unserialize()
passing a SerialInfo, but Serializer::Unserialize() expects a bool -- is
that intended?

The reason why I'm looking at this is that I'm trying to find the right
knobs to tweak to allow arbitrary *local* client applications to feed
information into Bro (like a tuned sshd that can feed its events and
traffic to a local Bro) without reinventing the wheel ...

Thanks so much!

Best,
Christian.
-- 
________________________________________________________________________
                                          http://www.cl.cam.ac.uk/~cpk25
                                                    http://www.whoop.org