Search Engine Land came out yesterday saying it will be a page by page basis, so I guess that answers my question.
From Google:
As we mentioned in this particular change, you either have a mobile friendly page or not. It is based on the criteria we mentioned earlier, which are small font sizes, your tap targets/links to your buttons are too close together, readable content and your viewpoint. So if you have all of those and your site is mobile friendly then you benefit from the ranking change.
But as we mentioned earlier, there are over 200 different factors that determine ranking so we can’t just give you a yes or no answer with this. It depends on all the other attributes of your site, weather it is providing a great user experience or not. That is the same with desktop search, not isolated with mobile search.