annotate HACKING @ 6687:d524232faed8

[gaim-migrate @ 7213] Nandan Dixit writes: " This patch has huge improvements for the yahoo protocol in gaim. Among the things it fixes are, Offline message receiving support. Proper support for buddy list (lists greater than 1024 chars) SMS and away embles on custom status" committer: Tailor Script <[e-mail]>

A lot of people have tried to hack gaim, but haven't been able to because
the code is just so horrid. Well, the code isn't getting better anytime
soon (I hate GNU indent), so to help all you would-be hackers help out
gaim, here's a brief tutorial on how gaim works. I'll quickly describe
the logical flow of things, then what you'll find in each of the source
files. As an added bonus, I'll try and describe as best I can how multiple
connections and multiple protocols work. Depending on how much I want to
avoid my final tomorrow I may even describe other parts of gaim that I
particularly want to brag about. Hopefully that's enough to get most of
you going.

If you don't know how event-driven programs work, stop right now. Gaim
uses GTK+'s main loop (actually GLib's but I won't talk about how GTK
works) and uses GLib functions for timeouts and socket notification. If
you don't know GTK you should go learn that first.

If you're going to hack gaim, PLEASE, PLEASE PLEASE PLEASE send patches
against the absolute latest CVS. I get really annoyed when I get patches
against the last released version, especially since I don't usually have
a copy of it on my computer, and gaim tends to change a lot between
versions. (I sometimes get annoyed when they're against CVS from 3 days
ago, but can't complain because it's usually my fault that I haven't
looked at the patch yet.) To get gaim from CVS (if you haven't already),
run the following commands:

$ export CVSROOT=:pserver:[e-mail]:/cvsroot/gaim
$ cvs login (hit enter as the password)
$ cvs co gaim (you'll see it getting all of the files)
$ cd gaim
$ ./autogen.sh

You'll now have your normal gaim tree with ./configure and all (which
./autogen.sh takes the liberty of running for you). (If you want to make
your life really simple, learn how CVS works. CVS is your friend.) To make
a patch, just edit the files right there in that tree (don't bother with
two trees, or even two copies of the same file). Then when you're ready to
make your patch, simply run 'cvs diff -u >my.patch' and send it off;
either post it on sf.net/projects/gaim in the patches section, or email it
to [e-mail].

This file was last modified by $Author: lschiere $ on
$Date: 2003-02-18 09:36:15 -0500 (Tue, 18 Feb 2003) $. Do not expect any information contained
within to be current or correct.

Here's something new. Someone requested that I comment the code. No. I'm a
lazy bastard, and I understand most of the code, so I don't need the
comments. I understand that some of you do though. So give me the names of
specific functions that you'd like commented and I'll see what I can do.
It's more likely that those comments will be updated with the code than
this file is, though even that is still unlikely.


CODING STYLE
============

Coding styles are like assholes, everyone has one and no one likes anyone
elses. This is mine and if you want me to accept a patch from you without
getting annoyed you'll follow this coding style. :)

It would probably just be easier for me to include CodingStyle from the
linux kernel source.

Tab indents. I *HATE* 2-space indents, and I strongly dislike 8-space
indents. Use a tab character. I'm likely to refuse a patch if it has
2-space indents.

K&R style for braces. Braces always go on the same line as the if, etc.
that they're associated with; the only exception is functions. Braces
for else statements should have both braces on the same line as the else
(i.e. "} else {").

No functionOrVariableNamesLikeThis. Save it for Java. Underscores are your
friend. "tmp" is an excellent variable name. Hungarian style will not be
tolerated. Go back to Microsoft.

I have a 105-char wide Eterm. Deal with it.

NO goto. I'm very likely to refuse a patch if it makes use of goto. If you
feel the need to use goto, you need to rethink your design and flow.


PROGRAM FLOW
============

Before gaim does anything you can see, it initializes itself, which is
mostly just reading .gaimrc (handled by the functions in gaimrc.c) and
parsing command-line options. It then draws the login window by calling
show_login, and waits for input.

At the login window, when "Accounts" is clicked, account_editor() is
called. This then displays all of the users and various information
about them. (Don't ask about what happens when "Sign On" is called. It's
quite hackish. The only reason the login window is there anymore is to
make it more palatable to people so used to WinAIM that they can't accept
anything else.)

When the "Sign on/off" button is clicked, serv_login is passed the
username and the password for the account. If the password length is
zero (the password field is a character array rather than pointer so it
will not be NULL) then the Signon callback will prompt for the password
before calling serv_login. serv_login then signs in the user using the
appropriate protocol.

After you're signed in, Gaim draws the buddy list by calling
show_buddy_list. Assuming the user has a buddy list (all buddy list
functions are controlled by list.c; when you sign on do_import is called
and that loads the locally saved list), the protocol calls
serv_got_update, which sets the information in the appropriate struct
buddy and then passes it off to set_buddy.

set_buddy is responsible for a lot of stuff, but most of it is done
implicitly. It's responsible for the sounds (which is just a call to
play_sound), but the biggest thing it does is call new_group_show and
new_buddy_show if necessary. There's only one group_show per group name,
even between connections, and only one buddy_show per group_show per
buddy name, even between connections. (If that's not confusing enough,
wait until I really start describing how the buddy list works.)

New connections happen the exact same way as described above. Each
gaim_account can have one gaim_connection associated with it. gaim_account
and gaim_connection both have a protocol field. This is kind of confusing:
gaim, except for the account editor screen and when the user signs on,
ignores the user's protocl field, and only uses the connection's protocol
field. You can change the connection's protocol field once it's created
and been assigned a PRPL to use to change certain behavior (Oscar does
this because it handles both AIM and ICQ). I'll talk about the
gaim_connection struct more later.

When the user opens a new conversation window, new_conversation is called.
That's easy enough. If there isn't a conversation with the person already
open (checked by calling find_conversation), show_conv is called to
create the new window. All sorts of neat things happen there, but it's
mostly drawing the window. show_conv is the best place to edit the UI.

That's pretty much it for the quick tutorial. I know it wasn't much but
it's enough to get you started. Make sure you know GTK before you get too
involved. Most of the back-end stuff is pretty basic; most of gaim is GTK.


SOURCE FILES
============

about.c:
  Not much to say here, just a few basic functions.

away.c:
  This takes care of most of the away stuff: setting the away message
  (do_away_message); coming back (do_im_back); drawing the away window;
  etc. Away messages work really oddly due to multiple connections and
  multiple protocols; I think there are really only two or three people
  who know how it works and I don't think any of us know why it works
  that way.

browser.c:
  Code for opening a browser window. Most of the code is trying to deal
  with Netscape. The most important function here is open_url. Have fun.

buddy.c:
  This takes care of the buddy list window and most things related to it.
  It still has some functions that manage the list, but not many.

buddy_chat.c:
  This takes care of the buddy chat stuff. This used to be a lot bigger
  until the chat and IM windows got merged in the code. Now it mostly
  just takes care of chat-specific stuff, like ignoring people and
  keeping track of who's in the room. This is also where the chat window
  is created.

conversation.c:
  This is where most of the functions dealing with the IM and chat windows
  are hidden. It tries to abstract things as much as possible, but doesn't
  do a very good job. This is also where things like "Enter sends" and
  "Ctrl-{B/I/U/S}" options get carried out (look for send_callback). The
  chat and IM toolbar (with the B/I/U/S buttons) are both built from
  the same function, build_conv_toolbar.

core.c:
  This is the start of what will become the main() for gaim-core.

dialogs.c:
  A massive file with a lot of little utility functions. This is where all
  of those little dialog windows are created. Things like the warn dialog
  and the add buddy dialog are here. Not all of the dialogs in gaim are in
  this file, though. But most of them are. This is also where do_import
  is housed, to import buddy lists. (The actual buddy list parsing code
  is in util.c for winaim lists and buddy.c for gaim's own lists.)

gaimrc.c:
  This controls everything about the .gaimrc file. There's not really much
  to say about it; this is probably one of the better designed and easier
  to follow files in gaim. The important functions are towards the bottom.

gtkimhtml.c:
  This is gaim's HTML widget. It replaced the old widget, GtkHtml (which
  was different than GNOME's GtkHTML). It's self-contained (it doesn't
  use any of gaim's code) and is actually a separate project from gaim
  (but is maintained by Eric).

html.c:
  Don't ask my why this is called html.c. Most of it is just grab_url,
  which does like the name says; it downloads a URL to show in the
  GtkHTML widget.  http.c would be a more appropriate name, but that's OK.

idle.c:
  This file used to be entirely #if 0'd out of existance. However, thanks
  to some very generous people who submitted patches, this takes care of
  reporting idle time (imagine that). It's a pretty straight-forward file.
  This also takes care of the auto-away stuff.

list.c:
  This file contains all of the routines for managing buddy lists,
  including importing them from a file, saving them, adding and removing
  buddies and groups, etc.

main.c:
  This is where the main() function is. It takes care of a lot of the
  initialization stuff, and showing the login window. It's pretty tiny
  and there's not really much to edit in it. This has some of the most
  pointless functions, like gaim_setup, which optionally turns off sounds
  on signon. A lot of this file should actually be part of other files.

md5.c:
  Oscar, Yahoo, and MSN all require md5 hashing, so better to put it in
  the core than have the same thing in three different places.

module.c:
  This contains all of the plugin code, except for the UI. This is what
  actually loads the plugins, makes sure they're valid, has the code for
  setting up plugin event handlers, and contains the plugin_event method
  that gaim calls on events.

multi.c:
  This is the file that tries to take care of most of the major issues
  with multiple connections. The best function in here by far is the
  account_editor(). auto_login() is also in here (I'm just reading multi.h
  now...). account_editor is really the only function that the UI needs
  to be concerned with.

perl.c:
  This was basically copied straight from X-Chat through the power of
  the GPL.  Perl is the biggest, most confusing piece of C code I've ever
  seen in my life (and keep in mind I'm a gaim hacker). I have a basic
  idea of what's going on in it, but I couldn't tell you exactly. The
  top half sets up perl and tells it what's going on and the bottom half
  implements the AIM module.

prefs.c:
  The important function in here is build_prefs, but the most useful
  function is gaim_button. build_prefs draws the window, and calls
  gaim_button probably 30 or 40 times. (I don't really wanna run grep
  | wc to count.) This is where you add the toggle button for gaim
  preferences. It's very simple, and if you look at a couple of the
  calls to gaim_button you'll figure it out right away. The new prefs
  window uses a CList instead of a Notebook, and there's a pretty bad
  hack to get it to work. I won't tell you what though.

proxy.c:
  Adam (of libfaim glory) got bored one day and rewrote this file, so
  now everything actually works. The main function is proxy_connect,
  which figures out which proxy you want to use (if you want to use one
  at all) and passes off the data to the appropriate function. This file
  should be pretty straight-forward.

prpl.c:
  This file is what lets gaim dynamically load protocols, sort of. All
  of the actual dlopen(), dlsym() stuff is in module.c. But this contains
  all of the functions that the protocol plugin needs to call, and manages
  all of the protocols. It's a pretty simple file actually.

server.c:
  This is where all of the differentiation between the different protocols
  is done.  Nearly everything that's network related goes through here
  at one point or another. This has good things like serv_send_im and
  serv_got_update. Most of it should be pretty self-explanatory.

sound.c:
  The main function in this file is play_sound, which plays one of 8
  (maybe 9?) sounds based on preferences. All that the rest of the code
  should have to do is call play_sound(BUDDY_ARRIVE), for example, and
  this file will take care of determining if a sound should be played
  and which file should be played.

util.c:
  There's not really a lot of cohesion to this file; it's just a lot of
  stuff that happened to be thrown into it for no apparent reason. None
  of it is particularly tasty; it's all just utility functions. Just
  like the name says.

plugins/ticker/gtkticker.c:
  Syd, our resident GTK God, wrote a GtkWidget, GtkTicker. This is that
  widget. It's cool, and it's tiny. This is actually a really good example
  widget for those of you looking to write your own.

plugins/ticker/ticker.c:
  Syd is just so cool. I really can't get over it. He let me come
  visit him at Netscape one day, and I got to see all of their toys
  (don't worry, I'm under an NDA). Anyway, this file is for the buddy
  ticker. This is also a damn cool file because it's got all of the
  functions that you'd want right up at the top. Someday I want to be
  as cool as Syd.

For the PRPLs, the only protocol whose "main" gaim file isn't the same as
the name of the protocol is ICQ; for that it's gaim_icq.c. But ICQ is
deprecated and you should be using Oscar for ICQ anyway.

HOW BUDDY LISTS WORK
====================

The buddy list is a pain in the ass. Let me start off by saying that. The
most difficult part about getting gaim to do multiple connections was
the buddy list. In its current state it's very much like the UI for
0.10.x and earlier, which is what I was aiming for. However, the code
is completely different. And not much better.

There are two parts to the buddy list: the lists for the connections and
the Buddy List window. list.c contains code to manage the lists themselves
and buddy.c contains the code for the Buddy List window.

Each buddy needs to belong to a connection, it cannot belong to a
"protocol" like in EveryBuddy. The reason is because when you are adding
buddies, you tell the server who is on your buddy list so it can tell you
about them; in order to tell the server, it needs to go out over a
connection. Going out over all connections would not be good, so you need
to specify which connection they go out on.

Managing lists is therefore fairly easy, each group and buddy has an
associated connection. Management functions like add_buddy/remove_buddy
and add_group/remove_group all take a gaim_connection. These are all in
list.c. They're boring.

The window is a lot more fun. There's really only one function that
does anything interesting, and that's set_buddy. (There's also things
like build_edit_tree, but that's boring.) set_buddy is called by
serv_got_update (and should only be called by that function) any time
a user signs on, signs off, goes away, comes back, goes idle, etc, etc,
etc. Various things happen depending on the new state of the buddy.

struct buddy has a member, present, which is set to either 0, 1, or
2. You can check if the buddy is online with "if (b->present)". This
becomes important. present is set to either 0 or 1 by serv_got_update,
or is not set at all. When the buddy is passed to set_buddy, if present
is 1 then set_buddy plays the BUDDY_ARRIVE sound, and sets present to 2,
to indicate it has already received notification of arrival. It then
does other signin-related stuff: setting the pixmap to the login icon;
updating the conversation windows; etc.

The most important thing it does though, if a buddy is present, is it
checks for the existance of the appropriate group_show and buddy_show for
that buddy.  Each buddy must belong to a group. group_shows are based on
name; there can only be one group_show for each group name. buddy_shows
are based both on name and on group_show; there can only be one buddy_show
in a group_show for each name. However, there can be two buddy_shows
with the same name as long as they have different group_shows.

Each buddy_show has a GList of connections that has registered its related
buddy as being online. set_buddy makes sure that the connection that it's
being passed is part of the connlist for the buddy_show associated with
the struct buddy that it's passed (it helps to know your data structures).

If a buddy logs off (b->present == 0), and a buddy_show exists for
that buddy, then set_buddy will play the logoff sound, change the icon,
remove the connection from the connlist for the buddy_show, etc.

And that's how that works. For the buddy lists, connections own buddies;
for the window, the buddies own the connections. When the buddy_show
connlist count drops to zero it disappears from existance.


PLUGINS
=======

OK, so you want to load a plugin. You go through whatever UI (you
can read all about the UI in plugins.c or whereever). You finally get
to load_plugin, the meat of the plugins stuff (plugins can actually
call load_plugin themselves to load other plugins). load_plugin
is passed the full path to the plugin you want to load
(e.g. /usr/local/lib/gaim/irc.so).

load_plugin does a few things with that filename. The first is to see
if you've already loaded that plugin. If you have, load_plugin unloads
the one that is currently loaded. You might wonder why; it's because
the same plugin can't be loaded twice. If you call g_module_open on a
filename twice, both times it will return the same pointer, and both times
increment the reference count on the GModule * that it returns. This
means you really do have the same plugin twice, which fucks up the
callback system to no end.  So it's better that you can only have it
loaded once at any given time.

Now that we're assured that we don't have this particular plugin loaded
yet, we better load it. g_module_open, baby. Much more portable than
dlopen().  In fact, for Linux it actually is the equivalent of dlopen()
(you can read the gmodule source and see for yourself). There's only one
quirk. It always logically ORs the options you pass with RTLD_GLOBAL,
which means that plugins share symbols. I haven't figured out yet if
this means just functions or variables too; but in either case make every
function and variable in your plugin static except for gaim_plugin_*(),
name(), and description().  It's good coding practice anyway.

So, assuming we didn't get NULL back from g_module_open, we then make sure
it's a valid gaim plugin by looking for and calling gaim_plugin_init,
courtesy g_module_symbol (g_module_symbol is actually what's portable
about gmodule as opposed to dl*; some BSD's require '_' prepended to
symbol names and g_module_symbol guarantees we do The Right Thing).

Assuming we've found gaim_plugin_init and it hasn't returned non-NULL
to us, we then add it to our list of plugins and go merrily about our way.

So when do the callbacks happen?! plugin_event, baby, plugin_event. Any
time you want to trigger a plugin event simply call plugin_even with the
parameters to be passed to any event handlers and you're set. plugin_event
then makes sure that any plugins waiting for the event get passed the
arguments properly and passes it on to perl.

Speaking of perl. If you really want to know how this works, you're
better off reading X-Chat's documentation of it, because it's better
than what I could provide.


MULTIPLE CONNECTIONS AND PRPLS
==============================

OK, let's start with the basics. There are users. Each user is contained
in an gaim_account struct, and kept track of in the gaim_accounts GSList.
Each gaim_account has certain features: a username, a password, and
user_info.  It also has certain options, and the protocol it uses to sign
on (kept as an int which is #define'd in prpl.h).

Now then, there are protocols that gaim knows about. Each protocol is
in a prpl struct and kept track of in the protocols GSList. The way the
management of the protocols is, there will only ever be one prpl per
numeric protocol. Each prpl defines a basic set of functions: login,
logout, send_im, etc. The prpl is responsible not only for handling
these functions, but also for calling the appropriate serv_got functions
(e.g. serv_got_update when a buddy comes online/goes offline/goes
idle/etc). It handles each of these on a per-connection basis.

So why's it called a PRPL? It stands for PRotocol PLugin. That means
that it's possible to dynamically add new protocols to gaim. However,
all protocols must be implemented the same way: by using a prpl struct
and being loaded, regardless of whether they are static or dynamic.

Here's how struct gaim_connection fits into all of this. At some point
the User (capitalized to indicate a person and not a name) will try to
sign on one of Their users. serv_login is then called for that user. It
searches for the prpl that is assigned to that user, and calls that prpl's
login function, passing it the gaim_account struct that is attempting to
sign on. The prpl is then responsible for seeing that the gaim_connection
is created (by calling new_gaim_connection), and registering it as
being online (by calling account_online and passing it the gaim_account and
gaim_connection structs). At that point, the gaim_account and gaim_connection
structs have pointers to each other, and the gaim_connection struct has
a pointer to the prpl struct that it is using. The gaim_connections are
stored in the connections GSList.  The way connection management works is,
there will always only be one gaim_connection per user, and the prpl that
the gaim_connection uses will be constant for the gaim_connection's life.

So at certain points the User is going to want to do certain things,
like send a message. They must send the message on a connection. So the UI
figures out which gaim_connection the User want to send a message on (for
our example), and calls serv_send_im, telling it which gaim_connection to
use, and the necessary information (who to send it to, etc). The serv_
function then calls the handler of the prpl of the connection for that
event (that was way too many prepositions). OK, each prpl has a send_im
function. Each connection has a prpl. so you call gc->prpl->send_im and
pass it the connection and all the necessary info. And that's how things
get done.

I hope some of that made sense. Looking back at it it makes absolutely no
sense to me. Thank god I wrote the code; otherwise I'm sure I'd be lost.


WRITING PRPLS
=============

Start off with a protocol that you want to implement; make sure it has a
number defined in prpl.h. If it doesn't, talk to Rob or Eric about adding
it. *NEVER* use an unassigned number, not even for testing or personal
use. It's possible that number will be used later by something else and
that would cause quite a few head-scratchers.

Start off with the following boiler plate:

static struct prpl *my_protocol = NULL;

void newproto_init(struct prpl *ret) {
	ret->protocol = PROTO_NEWPROTO;

	my_protocol = ret;
}

#ifndef STATIC

char *gaim_plugin_init(GModule *handle)
{
        load_protocol(newproto_init, sizeof(struct prpl));
        return NULL;
}

void gaim_plugin_remove()
{
        struct prpl *p = find_prpl(PROTO_NEWPROTO);
        if (p == my_protocol)
                unload_protocol(p);
}

char *name()
{
        return "New Protocol";
}

char *description()
{
        return PRPL_DESC("New Protocol");
}

#endif

Replace all NEWPROTO things with your protocol name (e.g. PROTO_OSCAR
instead of PROTO_NEWPROTO, oscar_init instead of newproto_init). Then
populate your struct prpl; the most important function is actually name(),
because without it, Gaim will most likely segfault. The second most
important function is login(). Not all functions need to be implemented.

There should be absolutely *ZERO* GTK in the PRPLs. PRPLs should *NEVER*
say what the UI *looks* like, only what information needs to be there.
There's currently an effort to get the GTK that is contained in the PRPLs
directory out of there. If you submit a patch that adds GTK to those
directories it's very likely to be refused, unless if I'm in a good mood
and decide to relocate things for you. That's not likely.

You're probably wondering how you can do certain things without GTK. Well,
you're just going to have to make do. Rely on the UI, that's why it's
there.  A PRPL should have absolutely ZERO interaction with the user, it
should all be handled by the UI.

Don't use the _options variables at all. The core should take care of all
of that. There are several proto_opt fields that you can use on a per-user
basis. Check out existing protocols for more details.