Well, if part of the "diversity" fair is socioeconomic (a lot of the ones that I've seen are), then you should be good.
It's not hard to bring up, just be proactive about it. Everyone will know that you're white (except any blind interviewers). A lot of them will probably wonder how you're diverse. So, you can bring up that you have a non-traditional background marked by crushing poverty, domestic violence, an imprisoned felon father, and that you raised your brother while you mom was working non-stop.
If you want you can state that you are diverse, but if they are looking for a limited sort of diversity that you understand and don't want to waste anyones time.
But really that's a pretty drastic life-story. I doubt that the vast bulk of law students have anything even approaching that. And that's diversity, isn't it? Good for you to rise above your circumstances of origin. I think the "over-coming bad youth" story is one that lots of people will receive favorably.