Thanks for your answer and guidance,
However in the plugin you link to the built-in folder delete tool doesn't ask for confirmation either.
But by reading the installation guide for the plugin I found out I could update my Side Bar's layout by commenting out or moving thing around. So I got rid of the "Delete Folder" option as I am not using it a lot.
Works for me.