The first hack that I am aware of is the Hello World hack by Nem. It was the first unsigned code to run on the PSP. It only ran on firmware version 1.00 PSPs, which were ones that had been imported from Japan. Later, the swap trick and then kxploit were developed for 1.50 firmwares, then the 2.00 TIFF exploit, the GTA:LCS exploits, more TIFF exploits, the Pandora, and today, another batch of TIFF exploits.
Of course, some of my details may be a little off. Others who are familiar with the scene, feel free to correct me.