If you have an iPhone, selecting this feature (voice isolation) which you can only turn on when on a call, really helps. Still not perfect but it helps.
Also I push my right hearing aid forward so the Mic is closer to my mouth, a bit.
Hope this picture helps.
Not sure how that would work with the tiny Audeo tho as I have the Naida.