"Remove" menuitem on sidebar is added by "SideBarEnhancements
" package, I'm the author of the package.
The behaviour you are requesting is not something I enjoy. I disagree with automated actions taken by the editor without my consentiment, such creating/opening/deleting /change focus/scroll things.
Since this can be easy customized via a preference I added : "close_affected_buffers_when_deleting_even_if_dirty" which is by default "False".
You need to update to latest version, to get this working.