Sublime Forum

Encoding UTF8 in plugins [OSX: python default ASCII] Solved!

#1

I was trying to use/adapt this code (http://mcgivrer.fr/sublime-text-2-the-killer-source-editor) in order to looking up on several websites such WordReference and others. If the selected text doesn’t have accents the code runs out of the box, but if selected text comes with accents or so (characters like ç, ã, ñ, ê…) it simply doesn’t work for me

import sublime, sublime_plugin, webbrowser
class OpenBrowserCommand(sublime_plugin.TextCommand):

        def run(self, edit, url):
                selection = ""
                for region in self.view.sel():
                        selection += self.view.substr(region)
                webbrowser.open_new_tab(url.format(search=selection))

I’m not a programmer (I’m just a common copywriter :unamused: ) but I tried to encode the string with several alternatives such as: u"", encode(string), string.encode(“utf-8”)… declaring encoding at the beginning of the script… without success.

Could you help with some tip I can use? I just want to create some text-filter (snippets for ReST and other text hijack: replacements, translations) and some web-lookup plugin like this. Maybe you can recommend me some page or example.

Thanks a lot!

0 Likes

#2

Work fine for me.

What’s the error in the console ?
And what’s the keybinding that call this command ?

0 Likes

#3

If you look for a specific package, look first at the Package Control Repository:
http://wbond.net/sublime_packages/community
A plugin like ‘Search Anywhere’ could be what your looking for.

Info on how to install PAckage Control is available here:
http://wbond.net/sublime_packages/package_control/installation

0 Likes

#4

[quote=“bizoo”]Work fine for me.

What’s the error in the console ?
And what’s the keybinding that call this command ?[/quote]

Hi Bizoo, thanks a lot for answering! :wink:

The console’s output is as follow:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 76: ordinal not in range(128)

And the example’s keybinding is:

{ "keys": "ctrl+alt+g"] , "command": "open_browser" , "args": {"url" : "http://www.google.fr/#hl=fr&output=search&sclient=psy-ab&q={search}&oq={search}"} }

In general settings it apears UTF-8 as default, so I don’t know the reason why it takes ascii as coding, so…

Thanks again.

0 Likes

#5

[quote=“bizoo”]If you look for a specific package, look first at the Package Control Repository:
http://wbond.net/sublime_packages/community
A plugin like ‘Search Anywhere’ could be what your looking for.

Info on how to install PAckage Control is available here:
http://wbond.net/sublime_packages/package_control/installation[/quote]

Hey, I didn’t know this wonderful page! I’ll check it out later this afternoon :wink:
Anyway I have to test if my issues with encoding is solved… :frowning:

Absolutely great :smile:

Thanks a lot!

0 Likes

#6

Hi again,

I tried Search anywhere as suggested but it occurs the same error:

File "./sublime_plugin.py", line 350, in run_ File "./google_search.py", line 13, in run File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/webbrowser.py", line 61, in open if browser.open(url, new, autoraise): File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/webbrowser.py", line 598, in open osapipe.write(script) UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 48: ordinal not in range(128)

Any idea?

It ocurrs in OSX (Snow Leopard), in Ubuntu 12.04 works perfect!
Default encoding in Terminal and Sublime: UTF-8

0 Likes

#7

I suppose your Python defaultencoding is ASCII (it’s the default for a new Python installation).
You can try to type in the console:

sys.getdefaultencoding()

The url you give as parameter to webbrowser.open_new_tab() is an unicode string.
But the instruction osapipe.write(script) need a standard string so Python try to convert your url to string using the defaultencoding (ASCII) which don’t work.

I suppose something like that could work:

def run(self, edit, url): selection = "" for region in self.view.sel(): selection += self.view.substr(region) webbrowser.open_new_tab(url.format(search=selection).encode('utf8'))
But it will probably only work for OSX and break Windows and linux support.

0 Likes

#8

Bizoo, thanks a lot!

Your’re right!

>>> sys.getdefaultencoding() 'ascii'

So recoding the string as you posted works, of course:

webbrowser.open_new_tab(url.format(search=selection).encode('utf8'))

I tried several ways to encode it but my python skills are so poor (yet…) :wink:

Anyway, don’t you think it is better to change the Python installation instead of encoding all strings for each script?
Wich is the best alternative? I use Linux as well so and -as you said- it wouldn’t work… What’s your opinion about that?

Thanks for your help, it was very useful for me.

0 Likes

#9

My knowledge of OSX is next to nothing, so maybe someone else could help…

I don’t think changing the defaultencoding of Python is a good idea, it’s the task of the programmer to take care of encoding issue.
For this particular issue you can modify your plugin with something like (not tested):

url = url.format(search=selection) if sys.platform == 'darwin': url = url.encode('utf8') webbrowser.open_new_tab(url)

I think this issue is related to how webbrowser module handle the opening of url on OSX.

0 Likes

#10

For example, searching “Español” in google will **NOT **open this url:

google.com/search?q=Español

You need to URL encode the parameters, like this:

google.com/search?q=Espa%C3%B1ol

an example…

selected = “español”
url = “http://google.com/search?q={search}
import urllib
txt = urllib.quote(selected.encode(‘utf8’), ‘’)
url.replace(’{search}’, txt)

0 Likes

#11

@bizoo I’ll follow your advice and I’m going to read more about encoding with Python in order to proper handling of strings when I’m creating filters. You know, I’m just a writer looking for the best of both worlds: programming and linguistics though I’m not a programmer my aim is to know all about tools which can make my life easier… and common office suites doesn’t :wink: Thanks sincerely for your help.

@tito [Deduzco que entiendes castellano, hehe, un saludo. También habrás notado que mi inglés tiene toques característicos :stuck_out_tongue:]

I think I understand your explaining, just encoding properly strings! As I said before it’s time to review Python tutorials for beginners like me.

By the way I have a question for you, too:

In your example you are using an URL like this:

url = "http://google.com/search?q={search}"

Ok, it runs fine for me… but when I’m trying with this one, it doesn’t run:

http://translate.google.com/#en|es|{search}

It shows a 404 error: it can’t compose that URL, whereas if you write directly to the browser seems to work correctly.
How can I handle ‘#’? I tried several ways to encode it…

def run(self, edit, url): selection = "" for region in self.view.sel(): selection += self.view.substr(region) webbrowser.open_new_tab(url.format(search=selection).encode('utf8'))

{ "keys": "ctrl+alt+g"] , "command": "open_browser" , "args": {"url" : "http://translate.google.com/#en|es|{search}"} }

Gracias por tu ayuda! I’m learning tons of useful things :smile:

0 Likes

#12

Hola!

Mira, anda para “view - open console”

Ahi pone, para ir probando…

import webbrowser

A mi en windows7 esto me funciona.

webbrowser.open_new_tab('http://translate.google.com/#en|es|{search}')

Y esto

webbrowser.open_new_tab('http://translate.google.com/#en|es|Español')

Me carga todo roto:

translate.google.com/#en|es|Español

Con el ejemplo que te puse, funciona bien:

webbrowser.open_new_tab('http://translate.google.com/#en|es|Espa%C3%B1ol')

Capas el problema esta en como remplazas la variable.

Prueba con esto:

def run(self, edit, url):
                selection = ""
                for region in self.view.sel():
                        selection += self.view.substr(region)
                import urllib, webbrowser
                webbrowser.open_new_tab( url.replace('{search}', urllib.quote(selection.encode('utf8'), '') ))
0 Likes