Git rm -r does not work in submodule

Try changing the submodule as described in the git book

$ git rm -r vendor/html-minifier
$ git submodule add https://github.com/kangax/html-minifier.git vendor/html-minifier

But he fails followed by

rm 'vendor/html-minifier'
fatal: git rm: 'vendor/html-minifier': Is a directory