This brand new Electrohome HiFi system looks incredibly cool and I've wanted to mention it here for a while. But as the old saying goes, looks are only skin deep. I don't recommend it because you can get far better gear for much less.
If you're in the market for a budget turntable that offers decent quality with readily available replacement cartridges, check out the Audio-Technica AT-LP60 ($90 on Amazon). This belt-driven automatic table runs at 33 1/3 and 45 RPM and accepts affordable ATN3600L cartridges. It also includes a built-in phono preamp, just in case your modern amplifier doesn't offer a turntable input.
Finding an affordable amplifier on Craigslist or Kijiji is fairly easy these days, but it's hard to find a better new deal than the Lepai LP-2020A+ Class-T mini amplifier (under $30 on Amazon) that we've raved about in the past. I no longer recommend Dayton Audio B652 speakers because of drastic component cost-cutting, but Sony SS-B1000 bookshelf speakers are an excellent compact replacement for under $70.
So there you have it. For about the same price as an all-in-one Electrohome HiFi system, you can own a belt-driven Audio-Technica turntable with Sony speakers and a surprisingly good Tripath-based amplifier. And if you can get your hands on a second-hand amp and speakers, you can put together a sweet vinyl playback system for under $100.