There is no easy answer to your question. There are at least five different Federal court rulings on whether a site needs to follow the ADA. Court outcomes very due to the nature of the site, whether you also have brick and mortar stores, where the disabled person lives, where your site is located as well as other outside factors.
This is one of those areas where the law will not catch up to the Internet of things for quite some time. In the meantime, some e-commerce sites will be sued, some will win and some will lose.
I am sorry you are getting a "maybe yes, maybe no" answer to your question. It is hard for lawyers to predict what will happen when the courts in different jurisdictions are ruling in different fashions.
Meanwhile, you may have other issues you need to address with your site. I have written a legal guide about the steps to take to begin. I have linked it to this answer below.
You may want to discuss your situation with a lawyer in more detail. Most lawyers on Avvo offer a free phone consultation.