I think Jews being generally well off has a lot to do with it. But why are they well off? Jews apparently have a higher average IQ score than Whites, so that could be part of the reason.
And being a small minority with a lot of power and influence is bound to cause suspicion I suppose.