You're right!

Soap helps remove more bacteria and dirt, and in particular the fat-soluble one. Remember to rub your hands after applying soap in order to achieve a better cleaning effect.