I know it is an old question, but it deserves an updated answer.
Since the introduction of BIP37 (=SPV with bloom filters) a number of serious privacy vulnerabilities have been discovered. By using SPV + address bloom filters you are exposing all the addresses of your wallet with every Blockchain surveillance company.
The reason behind this near perfect privacy breach is the combination of network analysis and various clustering strategies.
I conducted some research on the topic, which you can check out here. This medium post includes every possible references on the topic to date, of which I'd like to highlight Nick Jonas's Privacy in BitcoinJ blog post/video presentation and research paper.
Solutions include using a full node, a full-SPV node or BIP157/BIP158 client side filtering.