After going through two sets of earbuds I can confirm that these are NOT designed to work as hands free.
Actually got this confirmed by Sony Support via email. 

In Swedish:
"...men mikrofonen är designad för den aktiva ljudisoleringen. Inte för att vara optimal vid telefon samtal..."
In English :

"...the microphone is designed for the active noise canceling. It's not designed to be optimal for phone calls".

There you have it!
This is going back and it's not ok that Sony hides this fact and says "Hands-free calling with HD Voice" on that package.