rehighlight request

Discuss anything related to product development
Post Reply
Posts: 108
Joined: Thu Jun 11 2020 12:18 pm

rehighlight request

Post by seasoned_geek »

While working on changes to Diamond I have encountered another time honored Qt tradition of abusing the main event loop. When you have a source file of several thousand lines the processing time for the regular expressions locks the UI for several seconds. If you have 5 or 6 of these files open when you exit and Diamond restores them it is quite a bit before you can actually "do" anything with the editor.

The ideal solution would be the ability to paint/perform screen updates from a different thread. (I tried. While there are no compilation or runtime errors, the screen updates just never happen.)

The second or close to ideal solution would be the ability to create parented children in different threads. Given this is a fork of Qt that is probably nigh on impossible.

I can see some compromises.

1) Have rehighlight() call processEvents() every 200 lines or so. With short files of say 500 lines, the existing logic gives a snappy response on this machine.
AMD® Fx(tm)-6100 six-core processor × 6
24Gig of RAM

I know, calling processEvents() is a cardinal sin, but when you are forced to do either I/O or big compute in the main event loop, it's the only solution.


Code: Select all

void QSyntaxHighlighter::rehighlight()
   if (!d->doc) {

   QTextCursor cursor(d->doc);
   d->[b]rehighlight(cursor, QTextCursor::End);[/b]
Expose as a public slot the rehighlight() private method that accepts a begin and end cursor.
Instead of calling the existing rehighlight one could create a thread that would have a method with a timer in it. Every so many milliseconds it would signal via queued connection this slot with a new pair of cursors. The thread could keep track of where it was in the document and when it was done.

I haven't dug deeper into the performance issue. I certainly haven't gone into the internal rehighlight code to see if there are other performance improvements. I suspect the regular expression processing is a bit of a drag as well. Works well for small things but suffers from dynamic memory allocation issues. I know the XML stuff in Qt 4.8 suffered horribly from that. Found that out the hard way on an embedded medical device. We shaved 4+ minutes off the boot time bulk allocating memory ourselves rather than letting the Qt internals nibble at it as needed.

rehighlightblock() must have some overhead I haven't taken the time to really track down because swamping the main event loop with queued one block events 25ms apart doesn't achieve the desired result either. The sweet spot seems to be just under 500 lines. Anything over that and you notice lag until it gets just plain bad. I'm going to have to rummage around in my old code and see if I can scrounge up a few 10,000+ line source files to really test this out . . . once I get done adding EDT keypad support.

Post Reply