In OSX, there is a setting to use a proxy in System Preferences, but then that setting will be applied to all programs (except FF in my case), which is not what I want. So today I googled if there were any new addons for Chrome that could do that, and lo and behold, here is the solution (but not from a addon...)
Type the following in a Terminal.app (or iTerm2, which is awesome)
open -a /Applications/Google\ Chrome.app --args --proxy-server=proxy.home.over.vpn:8888I saved that string as an alias in .bashrc for reuse.
Taken from http://hints.macworld.com/article.php?story=20100213001826236