I certainly don't know the answer to your question. My story is anecdotal and I have no idea how others have been able to handle getting off the drug. I will tell you this, the doctor who prescribed the meds did not believe the problems that I was having, and I am sure that the drug reps were telling him that that kind withdrawal problem was just preposterous. Of course, many doctors assume that their patients are idiots and shouldn't disagree with their leaned opinions. Of course it is possible that my reaction was psychosomatic, but I don't think so, and it really doesn't matter, the results were real to me.
I do have some other odd reactions to drugs, so I may be somewhat unique. For instance, Morphine or Codine drugs make it impossible for me to sleep. Antihistamines that make other drowsy has the same insomniac reaction. My adult son has the exact same issues, so I chalk it up to genetics. I used to have to fight with his pediatric doctors who thought they knew more about my son's reactions to the drugs than my wife and I who directly observed the reactions. We had one doctor who flat told us that we were wrong, that a certain medicine would make my son sleepy, not wire him out. That doc was full of shit.
Another clue may be that I may have a genetic abnormality is that am descended from a long line of alcoholics. As far back as my historical research can reach, (at least back to the time of the American Civil War) all of the men on my father's side have been drunks. We are also the variety of alcoholics that become mean rather than happy when drinking. My temper would even scare me when I drank. My brother and I both had the same issues, but we both just stopped drinking when we were in our 40s before it destroyed our lives.
So, what I am trying to say is trust yourself, do what feels right. You have lived in your body for a long time and, on many levels, understand it far better than your doctor. Doctors, although well trained, are, regardless of what they think, are not always right in the best way to treat you. It has been my experience over the last nearly 70 years that doctors are very quick to make a diagnosis, and very slow to change their minds even when the evidence indicates that they may have been wrong.