Depends on what you're reading, really. Any story with length will eventually get around to all of the different creatures. Short ones will pick and choose, usually the ones that fit the theme of the story or that the author thinks will draw viewers.
Dragons end up in every story. Phoenix are a close second. Unicorns should be rather common. Trolls/ogres are a good miniboss. Kobolds are sort of overused, but not nearly as much as goblins or elves. Anything the hero can one hit while still making the reader think they might drown in a horde of critters. Ratkin/rats are popular... Demons are overused, mostly because a lot of anime uses different monsters that all get translated to 'demon', so everyone just lumps them together. Angels less so?
Undead are overused because they're more of a template. Sure, you've seen bandits, giant rats, and dragons, but now they're Undead! And you're next!~ And that's scary!
Humans are overused for reasons I won't discuss.
Nereids are rare because nobody likes writing ocean stuff. Water is scary, hard to logic out and what not. So all water creatures are also rare. Slightly more common but still rare are Mermaids, Krakens, and turtles(for some reason?).
Goblins are definitely overused, probably because of Goblin Slayer sexualizing them(not really, but that's why they're popular), but also because of DnD. The same reasons get a lot of beastkin and monster girls. If you have a monster girl, it probably went through the lense of Monster Musume/MGE, DnD, or both.
So in order for a monster to be rare, it has to be obscure enough not to get a DnD incarnation, creepy/inhuman enough not to get lewded, and it needs to have themes that won't apply to stories most often. For example, a monster that only shows up on a certain holiday once a year, ugly as funk, and never made it into DnD or Harry Potter or any other popular* fantasy. One or two movies that nobody watched don't count.
I saw a lonely thread and thought I'd write a novel on it. No need to thank me.~