Is this a serious enough adverse reaction to my meds to go to the hospital?

You need to put in a call for your psychiatrist. He's the only one who can help you. He may recommend the er or help you slowly wean off of the antidepressant that you are on now. Never abruptly stop taking it on your own. Some antidepressants will work for some but may not for you. This obviously isn't the right one for you. Don't give up there is a right one for you.