It depends on how you're using it. Usually, people just say
"goat cheese" to mean cheese made from goat milk. If you're talking
about cheese belonging to a goat, you'd say "goat's cheese."
Why not? it seems we try to conquer the world and all it has to
offer. if we make cheese from cows, and we get milk from Goats, why
not try cheese? it only makes sense.