Memory sticks can be used for a variety of functions... You can use one:
To move files, programs etc from one computer to another.
As extra RAM to give your computer more memory to use.
As an MP3 player (some memory sticks are fitted with a head-phone socket)
Chat with our AI personalities
A Memory Stick is for the saving of user data, on Sony's consoles. The consoles themselves typically (PSP, PS1, PS2, etc) do not have a hard drive, and cannot store any data apart from some personal settings, and their own internal firmware. If you want to create game save files, or download from the PlayStation Network, or play MP3's and video files, then you need a memory stick to put them on.
Having things on a memory stick also makes it easier to carry the saved games and other personal data around.