If magic is fake then why do we try to protect against it?

I was wondering because as a Muslim I know that no one can do anything beyond the means of a human but Allah and creatures he allows *some* supernatural abilities to like Jinn and Angels etc. However - I am wondering why are their so many hadiths and things to protect you against witchcraft? Why should we even be scared of Magic and witchcraft? In my mind I thought all of this stuff was fake and all pretend and make believe to fool the eyes using visual tricks. They have no real power to harm anyone or do anything using magic if it was even real. So why are we trying to protect ourselves against it? Shouldn't we just brush them off as frauds and visual tricksters (my current position) or should I genuinely be scared of that stuff in my mind as something that is real and that can harm me.