I think you should probably do it the other way around: only change the color scheme for the type of file you're gonna be editing (.txt, probably?).
1) Open a file of the type you're going to be editing (like Example.txt)
2) Go to Peferences > Settings - More > Syntax Specific - User
3) Put something along these lines there:
- Code: Select all
"color_scheme": "Packages/Color Scheme - Default/Your Own Theme File Here.tmTheme"
That should leave the other file types (like .sublime-settings) alone and only apply the desired color scheme to .txt files. Note that this is simply creating a new .sublime-settings file for the .txt file type. This new settings file will be named after the .tmLanguage file that defines the syntax def for .txt files. That's how syntax-specific settings work for every other file type.