Time and money is less of an issue, we just want to do what is best for the site
That is a fantastic position. SEO is a long-term proposition. This thinking should guide your entire decision making process.
Some people have mentioned in the past that sending a reconsideration request could do more harm than good
I cannot comment on what "some people" have shared. I read a lot of SEO related articles and there is a high percentage of questionable and outright incorrect information shared. I would ask you exactly who shared the advice and in what context.
Here is what Matt Cutts has shared on this topic: http://www.youtube.com/watch?v=5rsWc78dits
My strong opinion on the matter is as follows:
1. If no manual actions have been taken on your site, Google auto-replies. Accordingly, there is no harm in asking.
2. Matt shared in a different video (sorry, I was unable to easily locate the link) that his team does not go looking for problems as a result of a Reconsideration Request. Based on my knowledge and experience, if you have a penalty on your site and you submit a Reconsideration Request asking if you have a penalty, a member of the spam team will likely just push a button and share the canned response Google offers for penalties of that type. A Google employee would not go searching your site looking for issues.
3. For 100% of clients, I submit a Reconsideration Request upon accepting them as clients. It has never once been a cause for concern on any level.
Running a website is the act of a business. You cannot run your business in fear, and there is no reason to fear any aspect of the Google Reconsideration Request process as a white hat SEO or site operator.
Is it worth just removing links with no reconsideration request? Or is that essential?
It is essential to submit a Reconsideration Request if you are manually penalized.
One final note. There are legitimate other opinions on this topic. I have tremendous respect for Dr. Pete and agree with his approach 99.9% of the time, but I do recall him sharing a different viewpoint on this topic suggesting site owners not to submit a Reconsideration Request unless they had reason to believe they were penalized. Even if that were the case, in your instance there is strong reason to believe a manual penalty may exist on your site. Submit the Reconsideration Request and find out. Knowing is better then not knowing.