Selection highlight and focus on WPF TextBox

For version 1.0 naracea uses WPF TextBox as a base for text editing control. It is simple, it does what is needed to accomplish all infinite undo buffer features and it is already in .NET framework. And in some aspects it really… errr… doesn’t meet expectations. Main problem with the control is that it is impossible to do any custom painting on it without huge amount of code and some dirty tricks (all implementations of custom painting on WPF TextBox set text and background to transparent and then do all the formatting and painting one more time).

For v1.0 I don’t need syntax highlighting and text collapsing etc. This will come later, when all the 1.0 work is done. But I want my nice find and replace which shows matched text as selection, and WPF TextBox fails even in this simple task.

If I remember it right, WinForms TextBox has feature which allows user to set whether selection highlight should be visible even when control lost focus. There is no such thing for WPF TextBox. On stackoverflow you can find some advices to pretend focus lost event was handled when it actually wasn’t, but this approach results in this:

When selection is made before TextBox loses focus, it stays there until focus is gained again, and selection made programmatically by setting SelectionStart and SelectionLength properties is not visible. And since drawing highlight rectangle would require all those dirty hacks, I was looking for other solution.

And here it is: I use adorners. I add AdornerDecorator around Grid which holds TextEditor (which is my own class derived from TextBox), and by overriding couple of got-focus/lost-focus methods I paint selection highlight over the TextEditor on adorner layer.

Here is my TextEditor (shorten for brevity):

	public partial class TextEditor : System.Windows.Controls.TextBox {
		public SelectionHighlightAdorner SelectionHighlightAdorner { get; set; }
		public bool ShouldSuppressSelectionHighlightAdorner { get; private set; }

		private void UpdateSelectionHighlightAdorner() {
			this.ShouldSuppressSelectionHighlightAdorner = false;
			this.SelectionHighlightAdorner.InvalidateVisual();
		}

		protected override void OnPropertyChanged(DependencyPropertyChangedEventArgs e) {
			base.OnPropertyChanged(e);
			if( this.SelectionHighlightAdorner != null ) {
				switch( e.Property.Name ) {
					case "FontFamily":
						this.ShouldSuppressSelectionHighlightAdorner = true;
						break;
					case "FontSize":
						this.ShouldSuppressSelectionHighlightAdorner = true;
						break;
					default:
						this.ShouldSuppressSelectionHighlightAdorner = false;
						break;
				}
				this.SelectionHighlightAdorner.InvalidateVisual();
			}
		}

		protected override void OnLostKeyboardFocus(KeyboardFocusChangedEventArgs e) {
			base.OnLostKeyboardFocus(e);
			this.UpdateSelectionHighlightAdorner();
		}

		protected override void OnGotKeyboardFocus(KeyboardFocusChangedEventArgs e) {
			base.OnGotKeyboardFocus(e);
			this.UpdateSelectionHighlightAdorner();
		}

		protected override void OnSelectionChanged(System.Windows.RoutedEventArgs e) {
			base.OnSelectionChanged(e);
			this.UpdateSelectionHighlightAdorner();
			e.Handled = true;
		}
	}

(Disabling FontFamily and FontSize is needed when ribbon preview on font is ongoing.)

The adorner itself:

	public class SelectionHighlightAdorner : Adorner {
		TextEditor _editor;

		public SelectionHighlightAdorner(TextEditor editor)
			: base(editor) {
			_editor = editor;
			_editor.SelectionHighlightAdorner = this;
		}

		protected override void OnRender(DrawingContext drawingContext) {
			if( _editor.SelectionLength > 0 && !_editor.IsKeyboardFocused && !_editor.ShouldSuppressSelectionHighlightAdorner ) {
				drawingContext.PushClip(new RectangleGeometry(new Rect(0, 0, _editor.ActualWidth, _editor.ActualHeight)));
				int firstCharIndex = _editor.SelectionStart;
				int lastCharIndex = firstCharIndex + _editor.SelectionLength;
				var firstCharRect = _editor.GetRectFromCharacterIndex(firstCharIndex);
				var lastCharRect = _editor.GetRectFromCharacterIndex(lastCharIndex);

				var highlightGeometry = new GeometryGroup();
				if( firstCharRect.Top == lastCharRect.Top ) {
					// single line selection
					highlightGeometry.Children.Add(new RectangleGeometry(new Rect(firstCharRect.TopLeft, lastCharRect.BottomRight)));
				} else {
					int firstVisibleLine = _editor.GetFirstVisibleLineIndex();
					int lastVisibleLine = _editor.GetLastVisibleLineIndex();
					if( _editor.GetLineIndexFromCharacterIndex(firstCharIndex) < firstVisibleLine ) {
						firstCharIndex = _editor.GetCharacterIndexFromLineIndex(firstVisibleLine - 1);
						firstCharRect = _editor.GetRectFromCharacterIndex(firstCharIndex);
					}
					if( _editor.GetLineIndexFromCharacterIndex(lastCharIndex) > lastVisibleLine ) {
						lastCharIndex = _editor.GetCharacterIndexFromLineIndex(lastVisibleLine + 1);
						lastCharRect = _editor.GetRectFromCharacterIndex(lastCharIndex);
					}

					var lineHeight = firstCharRect.Height;
					var lineCount = (int)Math.Round( (lastCharRect.Top - firstCharRect.Top ) / lineHeight);
					var lineLeft = firstCharRect.Left;
					var lineTop = firstCharRect.Top;
					var currentCharIndex = firstCharIndex;
					for( int i = 0; i <= lineCount; i++ ) {
						var lineIndex = _editor.GetLineIndexFromCharacterIndex(currentCharIndex);						
						var firstLineCharIndex = _editor.GetCharacterIndexFromLineIndex(lineIndex);
						var lineLength = _editor.GetLineLength(lineIndex);
						var lastLineCharIndex = firstLineCharIndex + lineLength - 1;
						if( lastLineCharIndex > lastCharIndex ) {
							lastLineCharIndex = lastCharIndex;
						}
						var lastLineCharRect = _editor.GetRectFromCharacterIndex(lastLineCharIndex);
						var lineWidth = lastLineCharRect.Right - lineLeft;
						if( Math.Round(lineWidth) <= 0 ) {
							lineWidth = 5;
						}
						highlightGeometry.Children.Add(new RectangleGeometry(new Rect(lineLeft, lineTop, lineWidth, lineHeight)));
						currentCharIndex = firstLineCharIndex + lineLength;
						var nextLineFirstCharRect = _editor.GetRectFromCharacterIndex(currentCharIndex);
						lineLeft = nextLineFirstCharRect.Left;
						lineTop = nextLineFirstCharRect.Top;
					}
				}

				drawingContext.PushOpacity(0.4);
				drawingContext.DrawGeometry(SystemColors.HighlightBrush, null, highlightGeometry);
			}
		}
	}

This implementation does some optimizations on how much of text needs to be adorned and also reproduces original selection highlight as much as possible (including multiline selections).

Putting it together:

textEditorHolder.Children.Add(textEditor);
var layer = System.Windows.Documents.AdornerLayer.GetAdornerLayer(textEditor);
layer.Add(new SelectionHighlightAdorner(textEditor));

And the XAML snippet:

<AdornerDecorator>
	<Grid Name="textEditorHolder" MinWidth="100" HorizontalAlignment="Stretch"/>
</AdornerDecorator>

It is probably not the nicest solution, but until I replace WPF TextBox with something more powerful, it does the job pretty well:

(Not so) Ugly truth

There is no nice way to say it, so I’ll say it straight: naracea consumes memory. Lots of memory, considering it is a text editor. Yes. It is true.

Now, let me explain a bit. When I started working on the editor, I was considering how much memory usage is OK, and I did some calculations of how much RAM will it take to keep the whole history of longer document. After being afraid of the results for a while, I realized that the question is wrong. The right question is: what is the cost of losing some text for me, and the cost of rewriting it. And looking on the problem from this side, I’ve found, that couple of extra megabytes is really, really cheap.

And this is what I’m thinking since then. The memory is cheap. It is so cheap, that you don’t need to think about it. You can buy couple of extra gigabytes of RAM for the price of the dinner, and similar is true for the disk space (well, it is not so true when thinking about SSD drives, but even those are getting cheaper as I type).

I’m regularly testing naracea’s performance on couple of large documents (mostly lorem ipsum with lots of copy & cut & paste as changes), and for my reference document (around 700 thousands of single character changes), it takes about 200MB of RAM to keep all the changes in memory. There is also some more memory consumed when history searches are ongoing, but for document of this size it doesn’t add too much to the overall memory footprint. And here is what I think: 200MB it is nothing. My computer has 8 gigs of RAM, pretty much every laptop you can buy today has at least 3GB of RAM. The only low-memory PCs today are some very cheap netbooks, and yes, people use them to do lots of on-the-road editing and writing, but even there 200MB is almost nothing. Right now I have a tab open in Chrome which uses twice as much memory and I do not find anything bad or scary about it.

So it seems that times when memory consumption was important are over.

On the other hand, saving the document to the file is a completely different story. From the very beginning I knew naracea will need its own proprietary file format. I’ve tried several different persistence strategies (I’ll write about the long and sad story of naracea’s file format evolution in one of the future posts), and what I ended up with for several reasons is XML. The XML it not known as the most succinct format on the planet, and files are very different from memory structures. Files tend to be moved around in emails, on USB sticks and so on, so for file it makes lot of sense to make is as small as possible. Therefore the whole document in XML is compressed while it is being saved, with the compression level set so it gives reasonable file sizes at minimum performance penalty.

So the saved reference document with 700k changes takes around 3.5MB of the disk space. That means approximately 5 bytes for one character in the editor. And I think that’s reasonable price for the features I get from the editor.

Find and replace

Designing find and replace for naracea was so much fun for me.

You know, I’ve always felt somehow dissatisfied with how searching works in most text editors. There are usually two things: a) find next/find previous and b) find all. And once you edit something in front of the text you have searched for, you must search again, because search results are not being updated. Let’s face it: every search engine struggling with limited browser abilities have better UI for search results than most of desktop text editors backed with all those fine tuned UI frameworks.

So after a short period when I was considering going the usual way, I decided to try to do something little more user friendly with search:

What we have here is seemingly the usual set of next/prev/all buttons, including some replace buttons (because I believe finding and replacing are related so much in text editing, that they should never be split in two windows).

As you can see, results contain a lot of additional information. There is text surrounding matched term, so it is easier to see which result is the right one and there is information about position in the text. Also document and branch where the match was found are there. And as the finishing touch, there is replace button on each result item, so you can replace the matched text right there, without moving the mouse pointer to the top of the window.

Also: searching for something else (or in some other document or document’s branch) doesn’t necessarily clear the results. It is possible to have listing of several searches in the results list. Clicking on the result will take you to document, branch and text position of the match.

But what happens if we insert or delete some characters in the text? Right, the search results get updated as the text changes, so even after you edited the document, the results are still pointing to that one occurrence of the text we have searched for.

This seems more like what I would like to have in all text editors with find and replace.

When this was finished, I’ve started thinking how timeline, while useful, is just not enough to make all those stored changes really useful. To be really able to find something in history of the document there needs to be a way to search it.

And here it is: by selecting “Search the text and its history” in combobox above the replace all and find all buttons, you can search the whole history of the document. Of course, it is not possible to replace anything, because as we know history cannot be changed, but using this feature you can easily find whatever text you’ve written though the whole existence of the document:

Of course, for long documents, it takes some time to go through all the changes and find all occurrences of the word or sentence, so the searching of the history runs asynchronously, and it is possible to write more text while the search is ongoing. The search always starts at the current change and the changes are rewound one by one and the rewound document is then searched for the search term.

So this is how search and replace works in naracea. There are couple of interesting scenarios which are enabled by the history searches, but I’ll keep these for future posts.

Beta is now open

Finally. After long wait, there is first beta of naracea available.

If you wan to participate in this first private beta programme, please, send me an email to jiri at naracea dot com.

When undo is not enough

Writing, at least for me, is iterative process. I tend to write something, then delete it only to find that I actually need deleted part of the text in other place. Eventually I realize, that what I’ve written first time was better than the version I use instead of it.

For short blog posts it is not that annoying, but when writing longer essays or fiction, I feel quite miserable from time to time, because I need to re-write something. To correct this, I decided, that if I was going to write text editor, it would have infinite undo, and that the undo buffer would be saved along with the text.

When I started prototyping what eventually become naracea, I realized two things. First of all, when you store all typing and deleting you have done, there is no need to store the actual text. You can reconstruct it from recorded changes, and with a bit of optimization on the UI side, very good performance can be achieved. Second thing is, that to manage infinite undo buffer old undo and redo operations are not enough.

Let’s take a look on how the undo usually works: we start typing some text:

Lorem ipsum dolor sit amet

Now assume we undo last three words:

Lorem ipsum

But what happens when we hit period? We get the text

Lorem ipsum.

and empty undo buffer. So now the text “dolor sit amet“ is lost, and there is no way getting it back. It is indeed logical behavior, because with just two commands for managing undo and redo, there is no way to get any better. And that’s the reason why naracea uses along with undo and redo (which work the very same way they do in all other text editors) also rewinding and replaying.

If you think about that, undo and redo are editing commands like typing or deleting. It actually alternates the text. But to be able to keep infinite history of the text, we need operations, which are not changing the text it self. We just need to move though the recorded changes and revoke the effect they have on the text.

So in naracea, using undo or redo will store the “change”, which points to changes which are being undone or redone. On the other hand using rewind and replay will just move though the stored changes and present the text in the form it was before each particular change was made.

So using our previous example, rewinding from

Lorem ipsum dolor sit amet

to

Lorem ipsum

will not store any change, or issue and editing command. By clicking on replay button we can get the original text back very easily.

What happens when we rewind the undo? Well, all undone changes will be restored, and the text will look like no undo was done at all.

So taking the final state of the undone document

Lorem ipsum.

and hitting the rewind button will change the text to

Lorem ipsum

Hitting replay again and again will change the text to

Lorem ipsum d

then to

Lorem ipsum do

and if we keep hitting the button long enough, we will end up with the text we had before pressing the first undo:

Lorem ipsum dolor sit amet

Now, what happens if we continue using rewind? The characters will start to disappear again, but now we are not undoing them. We just rewind all the changes until we end up with empty screen. But remember, all changes are stored in the application, so one easy hit on “replay all” button will bring the text back to the last change we have done:

Lorem ipsum.

(Of course, rewinding and replaying text by single change would be tedious, so there are convenience commands to rewind/replay by words, sentences and paragraphs, and similar convenience functions for undo, so you can undo the whole sentence at once.)

You might be thinking: what happens when I have rewound document and I press some key to insert a character in to the editor? Well, there are several options what to do (e.g. preventing editing on rewound text, or replaying it back to the end), but it seems most reasonable to just silently generate undo command, which will turn rewinding in to actual change and allow you to continue as if you undone all the text manually.

But there is more: if we have all these changes stored somewhere, what else can we do with them? You are right. We can try to generate some statistics out of them! While certainly it would be most interesting to build lots of charts and tables from the changes, for version 1.0 there is only one basic “statistic” available. Under the editing area, you can see the timeline. Timeline is graphical representation of how many changes you’ve done to the text in chosen time span.

It is possible to set the span from 1 minute to the 1 year, so you can analyze your writing habits and performance. By default application doesn’t show empty bars, which correspond to the time intervals you have written nothing, because it can turn really annoying really quickly, but for sure you can turn the empty bars on, and then you see your writing performance in one bar graph with resolution you like.

There are more distinctive features in naracea. I will shed some light on them in future posts.

Do we really need another text editor?

To make the long story short: Of course we do – there are never enough text editors!

When I was younger, I was writing a bit (don’t panic – all that happened in my mother language, which is kind of a Klingon dialect). I was writing essays here and there, I’ve written several short stories, and I also managed to write something what could be called a “novel”. At some point, I refrained from the narrative writing, but I was still doing some blogging (which I’m not very good at lately, but hopefully I will be able to fix that).

I was writing on different computers, in various editors. I’ve used my good old Amiga 1200 for most of the fiction, later I had several different PCs with different text editors. I’ve understood quite early, that for writing, I need a text editor — I’m not able to use things like Word for writing. Problem with most text editors is that they are not really text editors. They are editors of the text files. And there is difference between editing the text, and editing the dumb text file.

When you write with your pen, you never really lose anything you write on the paper. Sure, you can strike sentences, paragraphs or even whole pages. You can take pages away and stuff them into the folder for later use. But the only way to get really rid of written text is to burn it in the thrash can in the cheap and dirty hotel room. Or something like that. On the other hand, on computer, you just hit delete, save, and after you close the editor, it is gone. There is no evidence of what you have written in the whole universe (well, that’s not exactly true, but I guess we can live with the approximation for the time being).

And the fact, that text editors are editors of the text files, was always kind of ruining my writing experience. Usually, when working on longer text, I end up writing pieces of the text out of order, or I decide that I don’t like something and delete it, only to find couple of hours later, that that pretty stupid, and that I want it back. And honestly – as far as I know, there still isn’t editor which would do what I need on the level I’d like it.

Sure, there is couple of editors integrating a version control pretty closely to writing process, but the problem is, that you can get only so far with history granularity when using tools like Subversion.

Then, around 2006 or 2004 I decided I want to write one more novel. (Honestly, after first two years when I had only first chapter and couple of research documents, I was joking, that it will take me longer to write it than it took Tolstoy to write War and Peace. And I’ll tell you: around the fourth year the joke stopped being funny.) And because I’ve learnt a thing or two about programming, I decided, that it is time to develop the text editor which will be worth to be used for story this epic.

And that’s what naracea is going to be. The text editor which remembers everything you write. No changes to the text will be ever lost. You will be able to move back and forth through the history of the document, deleting paragraphs at your will, because you will know that you can always get back to them.

And of course: it will be the text editor. Not programmer’s editor bended to text writing by adding “wrap lines” check box.

This blog will track course of development of the naracea (well, not exactly, because I’m actually working on it couple of – uhm – years now), and hopefully I will be able to bring the 1.0 version live in following 6 months.

At least for the sake of my novel, because, I must admit, I’m not Tolstoy.