But I think the fact that you acknowledge and realize that they are closer to "the edge" shows that their mixing shows a bit in their phenotype.
If I see a White person who to me looks "just White", they're just White.
But then I may see another person who is White but I may wonder "hmm, she got something else up in her?" I think this is the "edge" you're referring to. Not meeting expectations 100% of what a group "should" look like.
At the end of the day they'd still be White but I think some people just see that there may be other things thrown into their heritage besides one race/group.