It doesn't. There are stories where spirits are involved, but no magic whatsoever.
I've read before that the term Fantasy is really broad. In fact, if what I know still applies (do correct me if I'm wrong), there are a lot of subgenres in Fantasy, like...
1) High Fantasy--basically the setting is in another world. (Like the Tolkien Legendarium)
2) Low Fantasy--a story that is set on Earth, in a contemporary age. (Shakugan no Shana is an example)
3) Bangsian Fantasy--named after an author, it's a fantasy that involved famous dead people and their deeds after their death. (Just like in Drifters)
4) Lovecraftian Fantasy--I read this in some article a few years back; basically, stories with Lovecraft elements became separate from the standard fantasy subgenres due to many authors using and developing this trope.
5) Time-travel Fantasy--yeah, anything with time-travel stuff.
6) Sci-fi Fantasy--maybe this would be controversial (or many would disagree), but Science Fiction can also be considered 'fantasy', though it's been well-developed that it can (and did) become a genre of its own.
7) Supernatural Fantasy--a subgenre that often involves spiritualism, and the border between magic and the supernatural is blurred.
I forgot the other 'established' sub-genres, so feel free to add them.
Also, if I may add (based on what I observed):
> Isekai Fantasy--a popular fantasy 'subgenre' nowadays, where the premise is someone from contemporary Earth being transported to another world. Can be further subdivided into 'power fantasy', 'villainess', 'revenge fantasy', 'wish-fulfillment', 'survival game', and 'reverse isekai'.
Well, as someone who's involved in teaching literature to students, I believe genres--like languages--are dynamic. Those terms don't stay in their definitions for too long, and sometime later, it will evolve into several other forms. A good example of this is my above-mentioned example, Isekai. Back in early 2000s, 'isekai' is limited maybe only to the Japanese, since it's their term for another world. Then, when the light novel, Zero no Tsukaima (The Familiar of Zero) came out (and became popular to my generation; I'm already 33, btw), a lot of authors were inspired to write stories where the MC got summoned into another world, possessing and using modern Earth knowledge to their advantage.
Hence the spread of the word, 'Isekai'.