Math fonts seem to be a little tricky. The package fontspec can only do so much – the rendering is usually unsatisfactory, if it works at all. The reason for this seems to be that math fonts need special treatment in TeX, something that does not seem to be trivial.

Another option is loading specific packages for math fonts, see this guide and this mailing list or sfmath for sans serif fonts.

A somewhat satisfactory solution seems to be using unicode-math in combination with math-compatible OpenType fonts such as the STIX fonts (see also this article, this post and this Typophile forum post). The problem with this solution is that \setmathfont{STIXGeneral} seems to conflict with the amsmath package, they redefine each other's definitions.

The pre-packaged fonts seem to work best, although this limits your flexibility. STIX fonts should in principle provide a solution for this, but I haven't gotten it to work satisfactorily yet.