Is there an online website where we can play with Underscore or Handlebar templates online without having to download anything?
if underscores are enough for your needs: http://rowno.imtqy.com/architect/
The Cloud9 IDE has a node.js environment that you can use to install template modules and run them as well, all from a browser.