I worked with a Federal Public Defenders' office last summer (my 1L summer). I think it would be comparable to a county office, with the big differences being the workload (county PD's get hammered), and the experience level (federal PD's are generally a bit more veteran than local PD's). But both those things are, I think, good- as the more work, and the more help the attorneys need, the more hands on experience you get.
All told, it was a pretty great experience, and has paid off in a number of ways--
1.) the FPDs spent a lot of time in the courtroom, and I was able to tag along constantly. This might seem menial, but when it came time to do mock trials this spring, I really stood out because of my familiarity with the courtroom.
2.) these guys were pretty veteran attorneys, but they have a heavy workload and need a lot of help. I got to do a lot of real work, not just writing stupid memos that don't get read, but drafting some pleadings and responses, really getting involved with individual cases.
3.) the experience has generally played very well in interviews for my 2L summer- because I got to do real work, I'm able to talk with some authority about really being helpful to the guys I worked for, and I've got a ton of stories and anecdotes to throw out for any situation.
THE ONE CAVEAT: I'm sort of leaning away from criminal law now, and am having to explain that "I wanted the best possible legal experience my 1L year,and got it w/FPD's, but I'm looking into other areas of law"-- this can be an annoying and awkward disclaimer, but most employers have been receptive and perfectly understanding that my interests have changed.