I buy CD's from many of the on-line vendors that Leo mentions (especially the Amazon Marketplace vendors), but also buy from the BMG Music club here in the States. BMG carries a good number of DG and EMI CD's several months after they are released, and the cost is generally around $7 USD per disc. I buy CD's from on-line vendors around the world (US, Japan, Germany, Britain, France, Canada, Australia).
I have downloaded a lot of free music over the years (mostly from Usenet -- if you want a primer
http://www.slyck.com/ng.php ) and also traded CD-R's. I generally like to own those recordings that I'll return to again, and make a habit of purchasing such ones that I have obtained from download or CD-R trade. In fact, I generally do not even look to my large collection of CD-R's, preferring to find the day's CD choices from my legitimate collection. Perhaps it is the old school record collector in me, but I like the feeling of seeing a real CD cover and booklet.
Recently, I joined eMusic and have been downloading legitimately. eMusic is a great deal for long-movement symphonies and concertos, and other such classical music (i.e. a 25' symphony movement is a single download, as is a 2'30" pop song -- you can see which is the better deal).
When I first started listening to classical music, I found that the main branch of the Houston library had a nice collection. The condition of the discs themselves was oft-times marginal though. Which reminds me: with the end of the year approaching, I really need to donate my many duplicate and/or excess discs to the library. I talk about doing this every year and never quite get around to doing it.