Is it permissible to refuse halal food?

I heard a friend say that as Muslims we should eat everything that has been made halal for us and if we don't then we will be questioned on the last day. I am specifically referring to rabbit meat here: I have never tasted it nor do I intend to, but is my refusal to eat it considered a sin on my part?