Of course that depends on each race. In the story I'm writing, the age of consent for elves is that 80 is marriageable age, but it is socially unacceptable for anyone over 100 to get together with someone between 80 and 100. Meanwhile, they are absolutely horrified at humans who consider their own species equivalent of age 40 to be an acceptable marriageable age.
(For the record, a 40 year old elf looks like a human 12 year old. An 80 year old elf is about matching a human 18 year old, and 100 is about equivalent to the human age 21.)
So, I largely take both culture and time-setting into account. IRL, back in medieval days, there was no such thing as an age of consent. In 1920, an age of consent of 12 was put into place in most nations around the world in order to put a stop to child prostitution. It later rose to 16 in most nations in the 50s and 60s as part of the feminist movement in order to put more emphasis on girls staying in school to be educated, and a significant minority rose it further to 18 each for their own reasons. (In the United States, whether the age of consent is 16 or 18 actually varies from state to state.)
That minority that rose the age of consent to 18 is significant enough that the internet regards 18 as the age of consent for all intents and purposes even though it is 16 for over 50% of the world, and around 20% of the world makes it 14, 20% makes it 18, and the remaining 10% have an age of consent that is equal to or lower than 12, assuming they have one at all. It's kinda shocking if you've grown up thinking the age of consent as 18 is the standard when in reality it's actually a minority that happens to be large enough to make the internet conform to the highest common benchmark in order to make everyone happy.
EDIT: (Oh, right, there's also this one odd country in Africa that makes the age of consent 21. Not exactly sure what the story is behind them being such an odd man out on this issue, especially when the rest of Africa represents some of the lower ages of consent.)
Anyway, knowing this is the history, I wasn't really brave enough to state there was no age of consent in human cities in my world but decided to go ahead and go with 12 for it because that's still shocking enough to accomplish my goal of solidifying the medieval mindset of the humans and somewhat sympathize with why the elves call the humans of this world barbaric. Meanwhile, elves frequently kidnap human children and use them as slaves. (Actually, they're regarded more as pets and given to elven children. Not certain which is more degrading.)
Basically, one of the themes of my story is that both sides do things that the reader would regard as horrific and you can sympathies with as well as denounce the practices of both sides just as easily. It's meant to be an ugly world, largely reflecting things from IRL history. Things as simple as these are really the minimum to include if you're going for this sort of theme and feel.