Sublime Forum

Best and most efficient way to perform a deferred action

#1

What would be the best and most efficient way for a plugin to perform an action “x” seconds after the caret has stopped moving? If the caret starts moving again before “x” seconds has expired, the action should not execute.

0 Likes

#2

That’s an important question, most get it wrong, or they just don’t notice that running something every time the caret moves is a very heavy thing to do.

I use this and other combinations:

[code]import time
import sublime_plugin, sublime
try:
import thread
except:
import _thread as thread

Pref = None

class DontMessUpWithModifiedListenersPlease(sublime_plugin.EventListener):

def on_selection_modified_async(self, view):
	if Pref.enabled and not view.settings().get('is_widget'):
		Pref.modified = True
		Pref.timing = time.time()

def my_action(self, view):
	now = time.time()
	if now - Pref.timing > Pref.run_every:
		Pref.timing = time.time()
		if Pref.modified and not Pref.running:
			Pref.modified = False

			Pref.running = True
			print('yeah') # hardcore action,<-- ideally this also should run in a thread to not block the UI, In that case.. you should also turn of the flag "Pref.running" to false once the thread completes, and not in the next line, as if this is threaded you can end with multiple threads running at the same time.
			Pref.running = False

def dmuwmlp_loop():
my_action = DontMessUpWithModifiedListenersPlease().my_action
while True:
my_action(sublime.active_window().active_view())
time.sleep(Pref.run_every)

def plugin_loaded():
global Pref
class Pref:
def load(self):
Pref.enabled = True
Pref.modified = False
Pref.run_every = 0.60
Pref.running = False
Pref.timing = time.time()
Pref = Pref()
Pref.load()
if not ‘running_dmuwmlp_loop’ in globals():
global running_dmuwmlp_loop
running_dmuwmlp_loop = True
thread.start_new_thread(dmuwmlp_loop, ())

if int(sublime.version()) < 3000:
plugin_loaded()[/code]

0 Likes

#3

If “print(‘yeah’)” is also a hardcore action, then you just start a thread there (to avoid blocking the ST UI), do all the needed computation, and when done, just set in the thread itself “Pref.running = False” to allow running it again. – I’m sorry for delay.

0 Likes

#4

My take:

[code]import time
import threading

import sublime_plugin

Time we should wait after edit ends, in seconds

timeout = 0.6

Global reference to our thread

selection_thread = None

class TimeoutThread(threading.Thread):
should_stop = False
last_poke = 0

def __init__(self, timeout, callback, sleep=0.1, *args, **kwargs):
    super(TimeoutThread, self).__init__(*args, **kwargs)

    self.timeout = timeout
    self.callback = callback
    self.sleep = sleep

def run(self):
    while not self.should_stop:
        now = time.time()
        if self.last_poke and (self.last_poke + self.timeout) < now:
            # Run the callback
            self.callback(*self.poke_args, **self.poke_kwargs)
            self.last_poke = 0

        time.sleep(self.sleep)

# Set a flag to signal that we want to terminate the selection_thread
def stop(self):
    self.should_stop = True

def poke(self, *args, **kwargs):
    self.last_poke = time.time()
    self.poke_args = args
    self.poke_kwargs = kwargs

def timeout_callback(view):
print(“do stuff on a view now”, view, view.id())

selection_thread = TimeoutThread(timeout, timeout_callback)

class SelectionListener(sublime_plugin.EventListener):

def on_selection_modified(self, view):
    if not view.settings().get('is_widget'):
        selection_thread.poke(view)

def plugin_loaded():
selection_thread.start()

def plugin_unloaded():
selection_thread.stop()
[/code]

I tried to keep it a bit more generic (as I always tend to do). I also thought about trying to save multiple timestamps for each different set of parameters (because currently the callback would not fire if you changed the selection on a different view within those 0.6 seconds), but then decided it’s not that useful and can be added later anyway if necessary.

Another thing to note is that ST3 added a new on_selection_modified_async method that will be called asyncronously, but it will not have the behaviour you describe and these implementations perform.

@tito: Your solution has mainly two flaws.

  1. The thread is never closed.
  2. You unnecessarily create a new instance of DontMessUpWithModifiedListenersPlease every 0.6 seconds.

Very late edit: My solution is actually flawed in a certain case but I didn’t bother to fix it. When the thread is poked with a different view while it’s awaiting the timeout of another one it gets overridden. There either needs to be a mechanic to identify different “callbacks” or a separate thread for each view, which is unfavorable. I guess I just need to load it with more features.

Followup edit: Here is a fixed version: : gist.github.com/FichteFoll/0694dadafe51eb493b47

0 Likes

#5

The first one is a feature not a bug, and the second… I don’'t mind.

Yours… :stuck_out_tongue:, as I’m reading maybe I’m wrong, it can run the same task even if the previous task is still running. See GitGutter slowing down the complete App. My version does not suffer that problem. So, I’ll recommend to stick to the first one. Which is easy to read, and efficient.

0 Likes

#6

I don’t understand that. I only create a single thread so how can I run multiple tasks at the same time?

It is definitely not as efficient since you unnecessarily create a new instance of a completely unrelated class for every tick, for no reason. This is a terrible design decision. My version just calls time.time() and evalualates some basic expression (which you do too, but more). The only thing that I do “worse” is not using on_selection_modified_async, probably because of my ST2 and ST3 combo-usage, but the two calls in there will hardly make a difference.

Regarding the easy to read part, the method my_action has no direct connection to your on_selection_modified and is instead called by a separate function, which pretty much only implements time.sleep. This is also one of the reasons why you need so many pseudo-global variables since you use them in many different places - and I think everyone knows that globals should not be used thoughtlessly. Furthermore, you use the globals() function directly where it is not necessary and I’ve never liked your Pref = Pref() construct (since it assigns a different type to the same identifier at a point in time that is not known beforehand).

I define a custom thread that is easily customized, which might look more confusing than yours on first sight, but once you spent a few seconds on it you will grasp what it does, assuming you know some Python. You can also re-use it easily because it uses an OOP style and is context-independant.

By the way,if you make a modification with your version and then quickly change the view within Pref.run_every seconds the action will actually run on the newly selected view instead of the original one where the event occured. (related)


Sorry if I went overboard here, but I was somewhat offended by your comment and had to defend myself and my decisions.

0 Likes

#7

Yeah, re-spawning a new DontMessUpWithModifiedListernersPlease probably isn’t the best way.

I have yet to play with on_selection_modified_async. All of my plugins were originally written to work with ST2, so when they were ported to ST3, the methodology didn’t change much for the sake of quick porting.

In the early days of BracketHighlighter, I know me and @tito went back and forth a bit on the final implementation of how to best handle the deferred bracket matching. @tito was the one who first made the pull request in BH, and it has been massaged into what it is today. BH wanted to execute not only on every time the caret moved, but on modification as well. If it didn’t, the brackets would not always be highlighted proper. But the idea, no matter how it is implemented is going to be the same.

  1. You need to track the desired events
  2. You need to determine when best to execute desired action (usually when you aren’t being bombarded with multiple events)
  3. Ideally do only what you need to do and nothing unnecessary during the looping process (so yeah, instantiating new classes is probably not ideal, it adds more overhead)

So BH does something similar to what @tito is suggesting with good results. Flags the events and when a sufficient amount of time has passed without other events, executes the payload (and do only what is necessary when idle). So this is a real world example that is currently being used. Since it is handling both caret moving and modifications, the handling of the events is a bit more complicated. It also restarts a fresh thread when the plugin is reloaded. There are many different ways to essentially do the same thing. Whether it is driven by Thread class as @FichteFoll shows, or the opposite like what @tito shows, as long as they follow the 3 main points, you are fine. I have used a variety of methods to thread stuff in different situations, and I believe my approachs overtime are evolving. If I were to redo BH’s threading would it look different…maybe, but it would still basically be doing exactly the same thing even if I packaged it differently. I am not sure about the performance of other’s plugins with regards to different methods, but I do know that BH has good performance in regards to deferring to ideal times to execute its payload.

BH is a beast that could probably use some more cleanup, but you can look at the source if you like: github.com/facelessuser/BracketHighlighter

But this is a real world example:
[pre=#232628] 191 class BhEventMgr(object):
192 “”"
193 Object to manage when bracket events should be launched.
194 “”"
195
196 @classmethod
197 def load(cls):
198 “”"
199 Initialize variables for determining
200 when to initiate a bracket matching event.
201 “”"
202
203 cls.wait_time = 0.12
204 cls.time = time()
205 cls.modified = False
206 cls.type = BH_MATCH_TYPE_SELECTION
207 cls.ignore_all = False
208
209 BhEventMgr.load()

1430 class BhListenerCommand(sublime_plugin.EventListener):
1431 “”"
1432 Manage when to kick off bracket matching.
1433 Try and reduce redundant requests by letting the
1434 background thread ensure certain needed match occurs
1435 “”"
1436
1437 def on_load(self, view):
1438 “”"
1439 Search brackets on view load.
1440 “”"
1441
1442 if self.ignore_event(view):
1443 return
1444 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1445 sublime.set_timeout(bh_run, 0)
1446
1447 def on_modified(self, view):
1448 “”"
1449 Update highlighted brackets when the text changes.
1450 “”"
1451
1452 if self.ignore_event(view):
1453 return
1454 BhEventMgr.type = BH_MATCH_TYPE_EDIT
1455 BhEventMgr.modified = True
1456 BhEventMgr.time = time()
1457
1458 def on_activated(self, view):
1459 “”"
1460 Highlight brackets when the view gains focus again.
1461 “”"
1462
1463 if self.ignore_event(view):
1464 return
1465 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1466 sublime.set_timeout(bh_run, 0)
1467
1468 def on_selection_modified(self, view):
1469 “”"
1470 Highlight brackets when the selections change.
1471 “”"
1472
1473 if self.ignore_event(view):
1474 return
1475 if BhEventMgr.type != BH_MATCH_TYPE_EDIT:
1476 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1477 now = time()
1478 if now - BhEventMgr.time > BhEventMgr.wait_time:
1479 sublime.set_timeout(bh_run, 0)
1480 else:
1481 BhEventMgr.modified = True
1482 BhEventMgr.time = now
1483
1484 def ignore_event(self, view):
1485 “”"
1486 Ignore request to highlight if the view is a widget,
1487 or if it is too soon to accept an event.
1488 “”"
1489
1490 return (view.settings().get(‘is_widget’) or BhEventMgr.ignore_all)

1493 def bh_run():
1494 “”"
1495 Kick off matching of brackets
1496 “”"
1497
1498 BhEventMgr.modified = False
1499 window = sublime.active_window()
1500 view = window.active_view() if window != None else None
1501 BhEventMgr.ignore_all = True
1502 bh_match(view, True if BhEventMgr.type == BH_MATCH_TYPE_EDIT else False)
1503 BhEventMgr.ignore_all = False
1504 BhEventMgr.time = time()

1507 def bh_loop():
1508 “”"
1509 Start thread that will ensure highlighting happens after a barage of events
1510 Initial highlight is instant, but subsequent events in close succession will
1511 be ignored and then accounted for with one match by this thread
1512 “”"
1513
1514 while not BhThreadMgr.restart:
1515 if BhEventMgr.modified == True and time() - BhEventMgr.time > BhEventMgr.wait_time:
1516 sublime.set_timeout(bh_run, 0)
1517 sleep(0.5)
1518
1519 if BhThreadMgr.restart:
1520 BhThreadMgr.restart = False
1521 sublime.set_timeout(lambda: thread.start_new_thread(bh_loop, ()), 0)

1524 def init_bh_match():
1525 global bh_match
1526 bh_match = BhCore().match
1527 bh_debug(“Match object loaded.”)

1530 def plugin_loaded():
1531 init_bh_match()

1538 if not ‘running_bh_loop’ in globals():
1539 global running_bh_loop
1540 running_bh_loop = True
1541 thread.start_new_thread(bh_loop, ())
1542 bh_debug(“Starting Thread”)
1543 else:
1544 bh_debug(“Restarting Thread”)
1545 BhThreadMgr.restart = True[/pre]

0 Likes

#8

I really like @facelessuser implementation :stuck_out_tongue: … and yes there are probably many ways to do it…

I understand and can appreciate these implementation are elegant, but these are failing at one important point. Performance. The “run” methods of these suggestions, If I’m reading correctly can STILL be running on a second tick.

Imagine… you are going to “tint” the gutter if the line you are on, is marked as modified by your… VCS… you call… run() … imagine it takes 1 second to resolve if the line has been modified… and you execute “run” 4 time per second… you have 4 running “runs”… The suggested implementation should track if the process is running before trying to running it again, as in item1

Agree, I’m unnecessary instantiating the class… removed :slight_smile:

0 Likes

#9

Agree, BH tries to resolve this by when entering run and setting “BhEventMgr.ignore_all = True” which causes all future events to be ignored until it gets set to false.

Now for blatant honesty:
Do I set “BhEventMgr.ignore_all = True” soon enough, meh. The run method runs on the main thread, so you won’t have have more than one method running simultaneously, at most you might have an additional one run right after, but I am not sure how often that would happen. Never cared to look into since performance is pretty good.

But here are the two things I could do better:

  1. set “BhEventMgr.ignore_all = True” on the background thread before calling the run method and release it when the run method is done on the main thread
  2. Lock access of the shared variables when accessing on main thread or background thread so the threads are not stepping on each others toes as shown below:

[pre=#232628]_LOCK = threading.Lock()
with _LOCK:
_RUNNING = True[/pre]

Both of these things have been on my mind and would probably improve things, but yeah, I just haven’t bothered with them yet.

0 Likes

#10

Maybe something more like this:
[pre=#232628] 16 LOCK = threading.Lock()

192 class BhEventMgr(object):
193 “”"
194 Object to manage when bracket events should be launched.
195 “”"
196
197 @classmethod
198 def load(cls):
199 “”"
200 Initialize variables for determining
201 when to initiate a bracket matching event.
202 “”"
203 with LOCK:
204 cls.wait_time = 0.12
205 cls.time = time()
206 cls.modified = False
207 cls.type = BH_MATCH_TYPE_SELECTION
208 cls.ignore_all = False
209
210 BhEventMgr.load()
211
212
213 class BhThreadMgr(object):
214 “”"
215 Object to help track when a new thread needs to be started.
216 “”"
217 with LOCK:
218 restart = False

1433 class BhListenerCommand(sublime_plugin.EventListener):
1434 “”"
1435 Manage when to kick off bracket matching.
1436 Try and reduce redundant requests by letting the
1437 background thread ensure certain needed match occurs
1438 “”"
1439
1440 def on_load(self, view):
1441 “”"
1442 Search brackets on view load.
1443 “”"
1444
1445 if self.ignore_event(view):
1446 return
1447 with LOCK:
1448 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1449 sublime.set_timeout(bh_run, 0)
1450
1451 def on_modified(self, view):
1452 “”"
1453 Update highlighted brackets when the text changes.
1454 “”"
1455
1456 if self.ignore_event(view):
1457 return
1458 with LOCK:
1459 BhEventMgr.type = BH_MATCH_TYPE_EDIT
1460 BhEventMgr.modified = True
1461 BhEventMgr.time = time()
1462
1463 def on_activated(self, view):
1464 “”"
1465 Highlight brackets when the view gains focus again.
1466 “”"
1467
1468 if self.ignore_event(view):
1469 return
1470 with LOCK:
1471 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1472 sublime.set_timeout(bh_run, 0)
1473
1474 def on_selection_modified(self, view):
1475 “”"
1476 Highlight brackets when the selections change.
1477 “”"
1478
1479 if self.ignore_event(view):
1480 return
1481 with LOCK:
1482 if BhEventMgr.type != BH_MATCH_TYPE_EDIT:
1483 BhEventMgr.type = BH_MATCH_TYPE_SELECTION
1484 now = time()
1485 if now - BhEventMgr.time > BhEventMgr.wait_time:
1486 sublime.set_timeout(bh_run, 0)
1487 else:
1488 BhEventMgr.modified = True
1489 BhEventMgr.time = now
1490
1491 def ignore_event(self, view):
1492 “”"
1493 Ignore request to highlight if the view is a widget,
1494 or if it is too soon to accept an event.
1495 “”"
1496
1497 return (view.settings().get(‘is_widget’) or BhEventMgr.ignore_all)
1498
1499
1500 def bh_run():
1501 “”"
1502 Kick off matching of brackets
1503 “”"
1504
1505 window = sublime.active_window()
1506 view = window.active_view() if window != None else None
1507 with LOCK:
1508 edit_type = BhEventMgr.type == BH_MATCH_TYPE_EDIT
1509 bh_match(view, edit_type)
1510 with LOCK:
1511 BhEventMgr.ignore_all = False
1512 BhEventMgr.time = time()
1513
1514
1515 def bh_loop():
1516 “”"
1517 Start thread that will ensure highlighting happens after a barage of events
1518 Initial highlight is instant, but subsequent events in close succession will
1519 be ignored and then accounted for with one match by this thread
1520 “”"
1521
1522 def should_restart():
1523 restart = False
1524 with LOCK:
1525 restart = BhThreadMgr.restart
1526 return restart
1527
1528 while not should_restart():
1529 with LOCK:
1530 if BhEventMgr.modified == True and time() - BhEventMgr.time > BhEventMgr.wait_time:
1531 BhEventMgr.ignore_all = True
1532 BhEventMgr.modified = False
1533 sublime.set_timeout(bh_run, 0)
1534 sleep(0.5)
1535
1536 if should_restart():
1537 with LOCK:
1538 BhThreadMgr.restart = False
1539 sublime.set_timeout(lambda: thread.start_new_thread(bh_loop, ()), 0)
1540
1541
1542 def init_bh_match():
1543 global bh_match
1544 bh_match = BhCore().match
1545 bh_debug(“Match object loaded.”)

1548 def plugin_loaded():
1549 init_bh_match()

1552 global HIGH_VISIBILITY
1553 if sublime.load_settings(“bh_core.sublime-settings”).get(‘high_visibility_enabled_by_default’, False):
1554 HIGH_VISIBILITY = True
1555
1556 if not ‘running_bh_loop’ in globals():
1557 global running_bh_loop
1558 running_bh_loop = True
1559 thread.start_new_thread(bh_loop, ())
1560 bh_debug(“Starting Thread”)
1561 else:
1562 bh_debug(“Restarting Thread”)
1563 with LOCK:
1564 BhThreadMgr.restart = True[/pre]

0 Likes

#11

I’m not sure how Lock works… I can’t help there… but if that block the main thread, then I’ll avoid it. Setting a flag should really just works. I’m witness btw, of the performance of complex and handy BH :slight_smile:

0 Likes

#12

In this scenario, you can see I haven’t been too concerned about it because I am not using them :smile:; BH works well enough. It is just a way to make things thread safe. Two threads accessing the same data can sometimes cause problems (not so much for read only data). If I was serious, I would just have to make sure where I need thread safety. Keep in mind I don’t claim to be a threading expert either. I was more just admitting that what I use isn’t perfect, but it works well enough.

0 Likes

#13

[quote=“tito”]
Imagine… you are going to “tint” the gutter if the line you are on, is marked as modified by your… VCS… you call… run() … imagine it takes 1 second to resolve if the line has been modified… and you execute “run” 4 time per second… you have 4 running “runs”… The suggested implementation should track if the process is running before trying to running it again, as in item1[/quote]

I think that you don’t know how exactly threads in the threading module work. You should check the docs for that.

The run() method is only run once, when I invoke start(). I then do the processing there and only there so the task itself can only run once at a time - as long as I don’t create another thread and run the poke method on it.

I do agree that resetting the timer once the task itself has finished might be useful at times, but this is dependant on the action and can be added later if necessary.

0 Likes

#14

The original question to the thread was, “Best and most efficient way to perform a deferred action” and that also include to keep track if things are still running. In you quote, I was refering to GitGutter that slowed down the app considerable because of the problem I described in the previous post.

0 Likes

#15

Thanks for the input, everyone. facelessuser’s method is something I’ve seen a few plugins employ, and seems to be the best solution available.

I do think though that this should be something natively available as part of the API, because:

  1. It’s something that is commonly needed.
  2. It’s something that is hard to get right, and there are bad implementations out there.
  3. It can probably be implemented very efficiently natively.

Might be something for the developers to consider.

0 Likes

#16
  1. I have written several plugins and didn’t need it once. You are right in that this is a reoccurring problem, but I wouldn’t say it’s common.
  2. I agree with that, but so are many other things. And maybe you want to implement it differently? You already saw three different solutions, each with their own (more or less) advantages and suitable for different needs. And now imagine how many other potentially useful code snippets there might be. Should a software with a powerful scripting language like Python really provide some random snippets for each possible use case? I think not. The “_async” are a great example of a useful abstraction of a common use case but this here is not as easily applicable.
  3. See 2.
0 Likes