Tim Ellis,
I will start this message by saying, that we have built a lot of custom WordPress sites lately, so we are a bit partial.
Saying that though, we have found that a lot of people like WordPress, because they have found it as an easier to use(content management system), especially once you start utilizing some of the advance features such as Gravity Forms.
From an SEO perspective, Wordpress allows for you to easy create pages and posts easily, as well as optimize each one of them for search. There are tons of SEO plug-ins such as Yoast, head-space, all-in-one SEO, and so on.
However, I tend to agree with everyone else on here if you are using Joomla and have no problems with it, why switch? If you're simply talking about making the switch from one CMS to another, just because someone said so...I would not do it.