You will definitely need sandpaper. For large repairs - a LOT of it! Multiple grits will be required.
Often, you will start with 40 grit to even out the damaged area and provide a regular shape to the repair area.
Depending on time between layups, you may need to sand with 80 to 100 grit between layers.
For finishing work, plan on working up to 220 grit for painted surfaces.
Gel-coat requires working up to 2000 grit for sprayed or brushed application.
When working on projects, I get my paper from a number of sources - based on convenience and cost.
Your neighborhood hardware store or home center can provide all the paper you need up to about 220 grit wet/dry at a reasonable cost.
For slightly higher grits (up to 1000 wet/dry), auto parts stores may be helpful (in small quantity assortments).
For the 1500 and 2000 grits, plan on finding a auto body paint store. The price is often reasonable, and if you have one nearby, count yourself lucky!
If you don't have an auto part store nearby -- OR prefer the convenience of having it delivered to your door, check out the 1500 grit 25 pack below.