Home Download Buy Blog Forum Support

sublime 2 is amazing... if you're not disabled

sublime 2 is amazing... if you're not disabled

Postby esjesjesj on Fri Apr 06, 2012 1:59 pm

I am leaving Emacs, after a lifetime of use, for sublime. maybe. I have an upper extremity disability and need to be able to drive every single bit of functionality within sublime from speech recognition. And then I need some functionality that isn't present but may be constructed from components of sublime. Ideally, I would like to be able to drive every bit of functionality programmatically, from Python. I would like to be able to read settings and data fields selected change grammar on the fly.

After that, I want to be able to create some new functionality to help me create and edit symbols in code. One of the biggest impediments to writing code using speech recognition is unpronounceable symbol names. The goal is to create a cache of names and the ability to swap between symbol form and spoken form so that one can edit the spoken form.

The help I need to do this as a mentor. You don't have to write code, you don't need to contribute to the design. All I ask is that you be there to answer questions while I work on my speech driven programming technique as the tool evolves. This shouldn't take a lot of time. It would be helping quite a few disabled programmers and simultaneously building a fantastic brand for sublime.

A bit of credentials: I have been disabled since 1994 and using speech recognition continuously since the. I was the organizer for programming by voice workshops circa 1999. I have developed a couple different models for speech recognition use that show promise. see http://blog.esjworks.com I haven't written up a full description of the idea I'm touching on here but, believe me, it has some disabled programmers besides myself very excited.

I also hope folks can see the advantage this tool would have to themselves. Somewhere between 20,000 and 60,000 developers (depending on whose numbers you use[1]) are injured every year, many of them permanently like me. their other disabling events temporary like broken bones or permanent like arthritis. Disabled developers typically don't "come to Jesus" until after they've been disabled and when they find there is no support, they have to walk away from programming as a career or risk being even more crippled.

any help I can get would be welcome. Climbing the learning curve quickly enough so that I can start being more effective writing code is a challenge but with your help, it's possible.

--- eric

[1] part of the problem of getting good numbers is that many corporations like Tyson have convinced many states to eliminate repetitive strain injuries from the workers comp coverage. The end result being that the injuries are no longer being tracked and workers are not getting any assistance when they are injured. Of course Tyson was only concerned about chicken plucker's but, programming, chicken plucking, who can tell the difference?
esjesjesj
 
Posts: 6
Joined: Fri Mar 16, 2012 6:56 am

Re: sublime 2 is amazing... if you're not disabled

Postby atomi on Sat Apr 07, 2012 11:20 pm

I think it's great that you're still coding.
If you could be more specific about your needs, I don't doubt you'll get some eager volunteers including me if I can help.
atomi
 
Posts: 342
Joined: Thu Jan 20, 2011 5:06 pm
Location: Los Angeles CA US

Re: sublime 2 is amazing... if you're not disabled

Postby sublimator on Sun Apr 08, 2012 5:06 am

likewise, happy to help out, but won't be for a week or so
It is better to remain silent and be thought a fool, than to speak out and remove all doubt
sublimator
 
Posts: 649
Joined: Thu Mar 20, 2008 5:41 am

Re: sublime 2 is amazing... if you're not disabled

Postby esjesjesj on Sat Apr 21, 2012 4:49 pm

https://docs.google.com/document/d/1In1 ... xWp4E/edit

this is the core of what I want to do.

http://blog.esjworks.com/post/2011/03/1 ... the-box-IV

Talks about some of the secondary navigation issues

Third is enabling nuance Select-and-Say. I have to PowerPoint presentations on this technology but unfortunately, I'm not allowed to upload PowerPoint files.

fourth is an RPC interface to all of the sublime functionality so that we can properly drive things from NaturallySpeaking and not just inject keystrokes.

thank you again for volunteering. I hope we can make this project work.
esjesjesj
 
Posts: 6
Joined: Fri Mar 16, 2012 6:56 am

Re: sublime 2 is amazing... if you're not disabled

Postby atomi on Sat Apr 21, 2012 6:42 pm

Your first request to translate spoken words and toggle them into symbols/variables is going to be quite an undertaking.
Effectively, the best solution in my opinion would be to use an existing spell check dictionary and create an algorithm that will translate those words into symbols. One idea would be to just remove vowels from the spoken word and concatenate multiple words together. For example Dragon Cat Count = drgnctcnt.

Unfortunately, in regards to navigation, I'm not familiar with the current tools used to navigate documents. I can only take you at your word and experience when you say landmarks, as you describe them, would be the ideal method of navigating code. I can see that certain keywords could be used to navigate to specific code but the problem lies in that multiple functions/classes/variables/arguments etc... exist in a single document. Possibly using keyword "navigate to line 22" or saying "navigate functions" and have a list of functions pop up - just some ideas.

As for RPC that is something you will need to speak to Jon (jps) the programmer for Sublime Text. I'm again, not familiar with Dragon from Nuance to tell if it allows you to hook into applications in a way that would improve coding and navigation. At first glance it seems like a lot of what you'd like to get done would have to be done through the Dragon software.
atomi
 
Posts: 342
Joined: Thu Jan 20, 2011 5:06 pm
Location: Los Angeles CA US

Re: sublime 2 is amazing... if you're not disabled

Postby esjesjesj on Sat Apr 21, 2012 10:11 pm

atomi wrote:Your first request to translate spoken words and toggle them into symbols/variables is going to be quite an undertaking.
Effectively, the best solution in my opinion would be to use an existing spell check dictionary and create an algorithm that will translate those words into symbols. One idea would be to just remove vowels from the spoken word and concatenate multiple words together. For example Dragon Cat Count = drgnctcnt.


http://blog.esjworks.com/post/2011/03/1 ... he-box-III
http://blog.esjworks.com/post/2011/03/1 ... -the-box-V

some related writings.

from the Google talk document I gave earlier,

"""The transformation happens because there is a database with both spoken name and string name forms. When there is no match in the database, the user is then asked to fill in the match. Obviously, in the beginning, there’ll be a lot of data entry of symbol names for spoken names; but over time it should improve to the point where you hardly ever enter a symbol name.
"""
and
"""This technique has one additional advantage. It gives the disabled developer the ability to work with a team’s coding standards and pre-existing symbols. Now disabled developers can integrate fully, both technically and socially, within a development team.
"""

It's really quite simple. The major components are the code to identify a spoken name/symbol name, look it up in a database and return a value. There are some minor complications with handling the initialization case but again read the document, it's all spelled out there. This is really a very simple concept/process.

one of the reasons I rejected the automatic translator out, at least at first, is because there are different conversion algorithms depending on the context in many languages and programming teams. The simple translation allows the user complete control over the translation. in the future, we can look at semi automatic translations but the mechanism of interacting with the user will be basic the same as the unknown case except the predictive form is used in place of unknown.

One of the problems with programming by speech has been overthinking the problem. What needs to happen is lots of trials and failures with simple stuff. I have no guarantees that this technique will work but I really need to try and live with it for a bit before I will know. Once that's done, combining it with the variable name creation also outlined my blog would make an interesting follow on experiment.


Unfortunately, in regards to navigation, I'm not familiar with the current tools used to navigate documents. I can only take you at your word and experience when you say landmarks, as you describe them, would be the ideal method of navigating code. I can see that certain keywords could be used to navigate to specific code but the problem lies in that multiple functions/classes/variables/arguments etc... exist in a single document. Possibly using keyword "navigate to line 22" or saying "navigate functions" and have a list of functions pop up - just some ideas.



Yes, folks have to listen to my experience in for big chunks of project and it would help if people start using speech recognition as well to understand the world I live in. :-) I'm sorry but you also need to live with my subjective evaluation of the technique when I live with it.

The problem with large numbers of functions variables etc. is something I'm quite aware of and is leading me towards the "disambiguation through reduction of scope" principle. One way to implement that is to look for the nearest feature that satisfies the query, not unlike find operation. Another technique is to find everything that's visible on the screen a friend and maybe the entire file) assign a little pop-up number and then the grammar would expect the user to say the number. The system would then perform the operation requested on the numbered item. But again, I want are to try the simple stuff first

As for RPC that is something you will need to speak to Jon (jps) the programmer for Sublime Text. I'm again, not familiar with Dragon from Nuance to tell if it allows you to hook into applications in a way that would improve coding and navigation. At first glance it seems like a lot of what you'd like to get done would have to be done through the Dragon software.


don't worry about that part. The grammar -- routines are something I have a lot of control over. I don't have control over is how to get at the internal buffers of the editor to do a "Select-and-Say" functionality.
esjesjesj
 
Posts: 6
Joined: Fri Mar 16, 2012 6:56 am

Re: sublime 2 is amazing... if you're not disabled

Postby weirdwarp on Sun Apr 14, 2013 12:13 pm

what happened to this discussion? I am a quadriplegic on a ventilator trying to program with sublime. It is not quite so sublime when you have to do it with voice! I am doing a course on Java with the open University and was an airline captain before that just in case you were wondering .

I can't afford Dragon professional 12 so I'm using Dragon NaturallySpeaking Preferred 12. It does an excellent job
nowadays. It is a clunky old program though and gets very confused with IDEs and text editors among other sorts
of programs. The main things are that it does not recognise snippets as words and also when you go to another program and return to sublime it does not recognise the words that you have put in previously.

Writemonkey is a good example of an excellent text editor for Dragon NaturallySpeaking. It is very quick and Dragon never seems to hang. There is even a way of putting in snippets but it lacks a lot of things that sublime does nicely. Colour coding is important to me for instance.

Any ideas as I seem to be going round in circles??
weirdwarp
 
Posts: 1
Joined: Sat Apr 13, 2013 1:56 pm


Return to General Discussion

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 16 guests