Home Download Buy Blog Forum Support

Best and most efficient way to perform a deferred action

Best and most efficient way to perform a deferred action

Postby tripshock on Wed Apr 30, 2014 2:40 am

What would be the best and most efficient way for a plugin to perform an action "x" seconds after the caret has stopped moving? If the caret starts moving again before "x" seconds has expired, the action should not execute.
tripshock
 
Posts: 7
Joined: Fri Aug 02, 2013 12:21 am

Re: Best and most efficient way to perform a deferred action

Postby tito on Sat May 03, 2014 9:04 am

That's an important question, most get it wrong, or they just don't notice that running something every time the caret moves is a very heavy thing to do.

I use this and other combinations:
Code: Select all
import time
import sublime_plugin, sublime
try:
   import thread
except:
   import _thread as thread

Pref = None

class DontMessUpWithModifiedListenersPlease(sublime_plugin.EventListener):

   def on_selection_modified_async(self, view):
      if Pref.enabled and not view.settings().get('is_widget'):
         Pref.modified = True
         Pref.timing = time.time()

   def my_action(self, view):
      now = time.time()
      if now - Pref.timing > Pref.run_every:
         Pref.timing = time.time()
         if Pref.modified and not Pref.running:
            Pref.modified = False

            Pref.running = True
            print('yeah') # hardcore action,<-- ideally this also should run in a thread to not block the UI, In that case.. you should also turn of the flag "Pref.running" to false once the thread completes, and not in the next line, as if this is threaded you can end with multiple threads running at the same time.
            Pref.running = False

def dmuwmlp_loop():
   my_action = DontMessUpWithModifiedListenersPlease().my_action
   while True:
      my_action(sublime.active_window().active_view())
      time.sleep(Pref.run_every)

def plugin_loaded():
   global Pref
   class Pref:
      def load(self):
         Pref.enabled   = True
         Pref.modified   = False
         Pref.run_every   = 0.60
         Pref.running    = False
         Pref.timing    = time.time()
   Pref = Pref()
   Pref.load()
   if not 'running_dmuwmlp_loop' in globals():
      global running_dmuwmlp_loop
      running_dmuwmlp_loop = True
      thread.start_new_thread(dmuwmlp_loop, ())

if int(sublime.version()) < 3000:
   plugin_loaded()
Last edited by tito on Sat May 17, 2014 3:10 am, edited 4 times in total.
Give APIs, let the community build the rest!
https://github.com/titoBouzout
tito
 
Posts: 855
Joined: Thu Sep 29, 2011 2:27 pm
Location: Montevideo, Uruguay

Re: Best and most efficient way to perform a deferred action

Postby tito on Sat May 03, 2014 9:10 am

If "print('yeah')" is also a hardcore action, then you just start a thread there (to avoid blocking the ST UI), do all the needed computation, and when done, just set in the thread itself "Pref.running = False" to allow running it again. -- I'm sorry for delay.
Give APIs, let the community build the rest!
https://github.com/titoBouzout
tito
 
Posts: 855
Joined: Thu Sep 29, 2011 2:27 pm
Location: Montevideo, Uruguay

Re: Best and most efficient way to perform a deferred action

Postby FichteFoll on Fri May 16, 2014 1:27 pm

My take:

Code: Select all
import time
import threading

import sublime_plugin

# Time we should wait after edit ends, in seconds
timeout = 0.6
# Global reference to our thread
selection_thread = None


class TimeoutThread(threading.Thread):
    should_stop = False
    last_poke = 0

    def __init__(self, timeout, callback, sleep=0.1, *args, **kwargs):
        super(TimeoutThread, self).__init__(*args, **kwargs)

        self.timeout = timeout
        self.callback = callback
        self.sleep = sleep

    def run(self):
        while not self.should_stop:
            now = time.time()
            if self.last_poke and (self.last_poke + self.timeout) < now:
                # Run the callback
                self.callback(*self.poke_args, **self.poke_kwargs)
                self.last_poke = 0

            time.sleep(self.sleep)

    # Set a flag to signal that we want to terminate the selection_thread
    def stop(self):
        self.should_stop = True

    def poke(self, *args, **kwargs):
        self.last_poke = time.time()
        self.poke_args = args
        self.poke_kwargs = kwargs


def timeout_callback(view):
    print("do stuff on a view now", view, view.id())

selection_thread = TimeoutThread(timeout, timeout_callback)


class SelectionListener(sublime_plugin.EventListener):

    def on_selection_modified(self, view):
        if not view.settings().get('is_widget'):
            selection_thread.poke(view)


def plugin_loaded():
    selection_thread.start()


def plugin_unloaded():
    selection_thread.stop()


I tried to keep it a bit more generic (as I always tend to do). I also thought about trying to save multiple timestamps for each different set of parameters (because currently the callback would not fire if you changed the selection on a different view within those 0.6 seconds), but then decided it's not that useful and can be added later anyway if necessary.

Another thing to note is that ST3 added a new `on_selection_modified_async` method that will be called asyncronously, but it will not have the behaviour you describe and these implementations perform.

@tito: Your solution has mainly two flaws.
1. The thread is never closed.
2. You unnecessarily create a new instance of DontMessUpWithModifiedListenersPlease every 0.6 seconds.
FichteFoll
 
Posts: 405
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Best and most efficient way to perform a deferred action

Postby tito on Sat May 17, 2014 1:12 am

The first one is a feature not a bug, and the second.. I don''t mind.

Yours... :P, as I'm reading maybe I'm wrong, it can run the same task even if the previous task is still running. See GitGutter slowing down the complete App. My version does not suffer that problem. So, I'll recommend to stick to the first one. Which is easy to read, and efficient.
Give APIs, let the community build the rest!
https://github.com/titoBouzout
tito
 
Posts: 855
Joined: Thu Sep 29, 2011 2:27 pm
Location: Montevideo, Uruguay

Re: Best and most efficient way to perform a deferred action

Postby FichteFoll on Sat May 17, 2014 2:12 am

tito wrote:Yours... :P, as I'm reading maybe I'm wrong, it can run the same task even if the previous task is still running.



I don't understand that. I only create a single thread so how can I run multiple tasks at the same time?

tito wrote:Which is easy to read, and efficient.


It is definitely not as efficient since you unnecessarily create a new instance of a completely unrelated class for every tick, for no reason. This is a terrible design decision. My version just calls time.time() and evalualates some basic expression (which you do too, but more). The only thing that I do "worse" is not using `on_selection_modified_async`, probably because of my ST2 and ST3 combo-usage, but the two calls in there will hardly make a difference.

Regarding the easy to read part, the method `my_action` has no direct connection to your `on_selection_modified` and is instead called by a separate function, which pretty much only implements `time.sleep`. This is also one of the reasons why you need so many pseudo-global variables since you use them in many different places - and I think everyone knows that globals should not be used thoughtlessly. Furthermore, you use the `globals()` function directly where it is not necessary and I've never liked your `Pref = Pref()` construct (since it assigns a different type to the same identifier at a point in time that is not known beforehand).

I define a custom thread that is easily customized, which might look more confusing than yours on first sight, but once you spent a few seconds on it you will grasp what it does, assuming you know some Python. You can also re-use it easily because it uses an OOP style and is context-independant.

By the way,if you make a modification with your version and then quickly change the view within `Pref.run_every` seconds the action will actually run on the newly selected view instead of the original one where the event occured. (related)

---

Sorry if I went overboard here, but I was somewhat offended by your comment and had to defend myself and my decisions.
FichteFoll
 
Posts: 405
Joined: Fri Mar 16, 2012 11:49 pm
Location: Germany

Re: Best and most efficient way to perform a deferred action

Postby facelessuser on Sat May 17, 2014 2:15 am

Yeah, re-spawning a new DontMessUpWithModifiedListernersPlease probably isn't the best way.

I have yet to play with `on_selection_modified_async`. All of my plugins were originally written to work with ST2, so when they were ported to ST3, the methodology didn't change much for the sake of quick porting.

In the early days of BracketHighlighter, I know me and @tito went back and forth a bit on the final implementation of how to best handle the deferred bracket matching. @tito was the one who first made the pull request in BH, and it has been massaged into what it is today. BH wanted to execute not only on every time the caret moved, but on modification as well. If it didn't, the brackets would not always be highlighted proper. But the idea, no matter how it is implemented is going to be the same.

1. You need to track the desired events
2. You need to determine when best to execute desired action (usually when you aren't being bombarded with multiple events)
3. Ideally do only what you need to do and nothing unnecessary during the looping process (so yeah, instantiating new classes is probably not ideal, it adds more overhead)

So BH does something similar to what @tito is suggesting with good results. Flags the events and when a sufficient amount of time has passed without other events, executes the payload (and do only what is necessary when idle). So this is a real world example that is currently being used. Since it is handling both caret moving and modifications, the handling of the events is a bit more complicated. It also restarts a fresh thread when the plugin is reloaded. There are many different ways to essentially do the same thing. Whether it is driven by Thread class as @FichteFoll shows, or the opposite like what @tito shows, as long as they follow the 3 main points, you are fine. I have used a variety of methods to thread stuff in different situations, and I believe my approachs overtime are evolving. If I were to redo BH's threading would it look different...maybe, but it would still basically be doing exactly the same thing even if I packaged it differently. I am not sure about the performance of other's plugins with regards to different methods, but I do know that BH has good performance in regards to deferring to ideal times to execute its payload.

BH is a beast that could probably use some more cleanup, but you can look at the source if you like: https://github.com/facelessuser/BracketHighlighter

But this is a real world example:
 191 class BhEventMgr(object):
192 """
193 Object to manage when bracket events should be launched.
194 """
195
196 @classmethod
197 def load(cls):
198 """
199 Initialize variables for determining
200 when to initiate a bracket matching event.
201 """
202
203 cls.wait_time = 0.12
204 cls.time = time()
205 cls.modified = False
206 cls.type = BH_MATCH_TYPE_SELECTION
207 cls.ignore_all = False
208
209 BhEventMgr.load()

...

1430 class BhListenerCommand(sublime_plugin.EventListener):
1431 """
1432 Manage when to kick off bracket matching.
1433 Try and reduce redundant requests by letting the
1434 background thread ensure certain needed match occurs
1435 """
1436
1437 def on_load(self, view):
1438 """
1439 Search brackets on view load.
1440 """
1441
1442 if self.ignore_event(view):
1443 return
1444 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1445 sublime.set_timeout(bh_run, 0)
1446
1447 def on_modified(self, view):
1448 """
1449 Update highlighted brackets when the text changes.
1450 """
1451
1452 if self.ignore_event(view):
1453 return
1454 BhEventMgr.type = BH_MATCH_TYPE_EDIT
1455 BhEventMgr.modified = True
1456 BhEventMgr.time = time()
1457
1458 def on_activated(self, view):
1459 """
1460 Highlight brackets when the view gains focus again.
1461 """
1462
1463 if self.ignore_event(view):
1464 return
1465 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1466 sublime.set_timeout(bh_run, 0)
1467
1468 def on_selection_modified(self, view):
1469 """
1470 Highlight brackets when the selections change.
1471 """
1472
1473 if self.ignore_event(view):
1474 return
1475 if BhEventMgr.type != BH_MATCH_TYPE_EDIT:
1476 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1477 now = time()
1478 if now - BhEventMgr.time > BhEventMgr.wait_time:
1479 sublime.set_timeout(bh_run, 0)
1480 else:
1481 BhEventMgr.modified = True
1482 BhEventMgr.time = now
1483
1484 def ignore_event(self, view):
1485 """
1486 Ignore request to highlight if the view is a widget,
1487 or if it is too soon to accept an event.
1488 """
1489
1490 return (view.settings().get('is_widget') or BhEventMgr.ignore_all)

...

1493 def bh_run():
1494 """
1495 Kick off matching of brackets
1496 """
1497
1498 BhEventMgr.modified = False
1499 window = sublime.active_window()
1500 view = window.active_view() if window != None else None
1501 BhEventMgr.ignore_all = True
1502 bh_match(view, True if BhEventMgr.type == BH_MATCH_TYPE_EDIT else False)
1503 BhEventMgr.ignore_all = False
1504 BhEventMgr.time = time()

...

1507 def bh_loop():
1508 """
1509 Start thread that will ensure highlighting happens after a barage of events
1510 Initial highlight is instant, but subsequent events in close succession will
1511 be ignored and then accounted for with one match by this thread
1512 """
1513
1514 while not BhThreadMgr.restart:
1515 if BhEventMgr.modified == True and time() - BhEventMgr.time > BhEventMgr.wait_time:
1516 sublime.set_timeout(bh_run, 0)
1517 sleep(0.5)
1518
1519 if BhThreadMgr.restart:
1520 BhThreadMgr.restart = False
1521 sublime.set_timeout(lambda: thread.start_new_thread(bh_loop, ()), 0)

...

1524 def init_bh_match():
1525 global bh_match
1526 bh_match = BhCore().match
1527 bh_debug("Match object loaded.")

...

1530 def plugin_loaded():
1531 init_bh_match()

...

1538 if not 'running_bh_loop' in globals():
1539 global running_bh_loop
1540 running_bh_loop = True
1541 thread.start_new_thread(bh_loop, ())
1542 bh_debug("Starting Thread")
1543 else:
1544 bh_debug("Restarting Thread")
1545 BhThreadMgr.restart = True
facelessuser
 
Posts: 1575
Joined: Tue Apr 05, 2011 7:38 pm

Re: Best and most efficient way to perform a deferred action

Postby tito on Sat May 17, 2014 2:43 am

I really like @facelessuser implementation :P .. and yes there are probably many ways to do it..

I understand and can appreciate these implementation are elegant, but these are failing at one important point. Performance. The "run" methods of these suggestions, If I'm reading correctly can STILL be running on a second tick.

Imagine.. you are going to "tint" the gutter if the line you are on, is marked as modified by your.. VCS.. you call.. run() .. imagine it takes 1 second to resolve if the line has been modified... and you execute "run" 4 time per second.. you have 4 running "runs"... The suggested implementation should track if the process is running before trying to running it again, as in item1

Agree, I'm unnecessary instantiating the class.. removed :)
Give APIs, let the community build the rest!
https://github.com/titoBouzout
tito
 
Posts: 855
Joined: Thu Sep 29, 2011 2:27 pm
Location: Montevideo, Uruguay

Re: Best and most efficient way to perform a deferred action

Postby facelessuser on Sat May 17, 2014 3:23 am

tito wrote:magine.. you are going to "tint" the gutter if the line you are on, is marked as modified by your.. VCS.. you call.. run() .. imagine it takes 1 second to resolve if the line has been modified... and you execute "run" 4 time per second.. you have 4 "running" "runs"... The suggested implementation should track if the process is running before trying to running it again,


Agree, BH tries to resolve this by when entering run and setting "BhEventMgr.ignore_all = True" which causes all future events to be ignored until it gets set to false.

Now for blatant honesty:
Do I set "BhEventMgr.ignore_all = True" soon enough, meh. The run method runs on the main thread, so you won't have have more than one method running simultaneously, at most you might have an additional one run right after, but I am not sure how often that would happen. Never cared to look into since performance is pretty good.

But here are the two things I could do better:
1. set "BhEventMgr.ignore_all = True" on the background thread before calling the run method and release it when the run method is done on the main thread
2. Lock access of the shared variables when accessing on main thread or background thread so the threads are not stepping on each others toes as shown below:

_LOCK = threading.Lock()
with _LOCK:
_RUNNING = True


Both of these things have been on my mind and would probably improve things, but yeah, I just haven't bothered with them yet.
facelessuser
 
Posts: 1575
Joined: Tue Apr 05, 2011 7:38 pm

Re: Best and most efficient way to perform a deferred action

Postby facelessuser on Sat May 17, 2014 3:42 am

Maybe something more like this:
 16 LOCK = threading.Lock()

...

192 class BhEventMgr(object):
193 """
194 Object to manage when bracket events should be launched.
195 """
196
197 @classmethod
198 def load(cls):
199 """
200 Initialize variables for determining
201 when to initiate a bracket matching event.
202 """
203 with LOCK:
204 cls.wait_time = 0.12
205 cls.time = time()
206 cls.modified = False
207 cls.type = BH_MATCH_TYPE_SELECTION
208 cls.ignore_all = False
209
210 BhEventMgr.load()
211
212
213 class BhThreadMgr(object):
214 """
215 Object to help track when a new thread needs to be started.
216 """
217 with LOCK:
218 restart = False

...

1433 class BhListenerCommand(sublime_plugin.EventListener):
1434 """
1435 Manage when to kick off bracket matching.
1436 Try and reduce redundant requests by letting the
1437 background thread ensure certain needed match occurs
1438 """
1439
1440 def on_load(self, view):
1441 """
1442 Search brackets on view load.
1443 """
1444
1445 if self.ignore_event(view):
1446 return
1447 with LOCK:
1448 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1449 sublime.set_timeout(bh_run, 0)
1450
1451 def on_modified(self, view):
1452 """
1453 Update highlighted brackets when the text changes.
1454 """
1455
1456 if self.ignore_event(view):
1457 return
1458 with LOCK:
1459 BhEventMgr.type = BH_MATCH_TYPE_EDIT
1460 BhEventMgr.modified = True
1461 BhEventMgr.time = time()
1462
1463 def on_activated(self, view):
1464 """
1465 Highlight brackets when the view gains focus again.
1466 """
1467
1468 if self.ignore_event(view):
1469 return
1470 with LOCK:
1471 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1472 sublime.set_timeout(bh_run, 0)
1473
1474 def on_selection_modified(self, view):
1475 """
1476 Highlight brackets when the selections change.
1477 """
1478
1479 if self.ignore_event(view):
1480 return
1481 with LOCK:
1482 if BhEventMgr.type != BH_MATCH_TYPE_EDIT:
1483 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1484 now = time()
1485 if now - BhEventMgr.time > BhEventMgr.wait_time:
1486 sublime.set_timeout(bh_run, 0)
1487 else:
1488 BhEventMgr.modified = True
1489 BhEventMgr.time = now
1490
1491 def ignore_event(self, view):
1492 """
1493 Ignore request to highlight if the view is a widget,
1494 or if it is too soon to accept an event.
1495 """
1496
1497 return (view.settings().get('is_widget') or BhEventMgr.ignore_all)
1498
1499
1500 def bh_run():
1501 """
1502 Kick off matching of brackets
1503 """
1504
1505 window = sublime.active_window()
1506 view = window.active_view() if window != None else None
1507 with LOCK:
1508 edit_type = BhEventMgr.type == BH_MATCH_TYPE_EDIT
1509 bh_match(view, edit_type)
1510 with LOCK:
1511 BhEventMgr.ignore_all = False
1512 BhEventMgr.time = time()
1513
1514
1515 def bh_loop():
1516 """
1517 Start thread that will ensure highlighting happens after a barage of events
1518 Initial highlight is instant, but subsequent events in close succession will
1519 be ignored and then accounted for with one match by this thread
1520 """
1521
1522 def should_restart():
1523 restart = False
1524 with LOCK:
1525 restart = BhThreadMgr.restart
1526 return restart
1527
1528 while not should_restart():
1529 with LOCK:
1530 if BhEventMgr.modified == True and time() - BhEventMgr.time > BhEventMgr.wait_time:
1531 BhEventMgr.ignore_all = True
1532 BhEventMgr.modified = False
1533 sublime.set_timeout(bh_run, 0)
1534 sleep(0.5)
1535
1536 if should_restart():
1537 with LOCK:
1538 BhThreadMgr.restart = False
1539 sublime.set_timeout(lambda: thread.start_new_thread(bh_loop, ()), 0)
1540
1541
1542 def init_bh_match():
1543 global bh_match
1544 bh_match = BhCore().match
1545 bh_debug("Match object loaded.")

...

1548 def plugin_loaded():
1549 init_bh_match()

...

1552 global HIGH_VISIBILITY
1553 if sublime.load_settings("bh_core.sublime-settings").get('high_visibility_enabled_by_default', False):
1554 HIGH_VISIBILITY = True
1555
1556 if not 'running_bh_loop' in globals():
1557 global running_bh_loop
1558 running_bh_loop = True
1559 thread.start_new_thread(bh_loop, ())
1560 bh_debug("Starting Thread")
1561 else:
1562 bh_debug("Restarting Thread")
1563 with LOCK:
1564 BhThreadMgr.restart = True
facelessuser
 
Posts: 1575
Joined: Tue Apr 05, 2011 7:38 pm

Next

Return to Plugin Development

Who is online

Users browsing this forum: No registered users and 5 guests