I suggest trying to get the new PSP Giga Pack that is out. Instead of the 32mb stick you get a 1GB memory stick for all your MP3's, videos, pictures and saves. Since 1GB memory sticks are still in retail stores hanging around the 100 buck mark and the Giga Pack is only 50 bucks more then the value pack (299 compared to 249) it's a much better deal.
Honestly I would not get the PSP for a younger kid though. They are pretty tough but the screen is not protected and easily scratchable. It's more of a handheld multimedia device then a kid's toy and is not built for kid punishment like the Gameboy SP's. Just keep this in mind. The DS is a little better for kids but my nephews bottom touch screen on his DS was scratched all up from the stylus within a month.