I would still tell your doctor you are worried about this and if they don't take it seriously I would suggest going to a doctor who would take it seriously. I hope everything works out for you!

Oops my bad I said that wrong I had a substance abuse problem in the past over 10 years ago

Tell the doctor you have the problem and hopefully they will fix it if not find a different doctor.

I have tried that one had allergic reaction to it

Try buspar.