The way I did it was to lock the bitmap with lockbits, then get at the data with marshal.readbyte, and set it again with marshal.writebyte. Since then, I've found that it is faster to define an array to hold the colors and use marshal.copy to move all the data into the array.
Attached is the pixelate example modified that way (vb2005, I've not got 2003 to hand). If you have 2003 then open the form1.vb and then copy and paste the code into a new project (you'll need to set up the form controls manually, and clean up any 2005 specific stuff).
To get it faster (vb is limited, so code tweaks will only get it a bit faster) you would need to move to c# or c++. See Christian Graus' articles which are linked to here for a bit on processing in c# http://www.bobpowell.net/imageprocessing.htm