Marian Schedenig's Website

During the four years since the first release of Grep Console, I frequently received requests for two features. The first was a way to export the Grep Console configuration and import it in another Eclipse workspace. That has been implemented in version 3 through the Load and Save buttons in the Grep Console dialog, as described in an earlier how-to.

Some background stuff

The most requested feature was a way to limit the console output only to those lines that match certain expressions, and disregard all the rest. In fact, when I set out to write the first version of Grep Console, that was exactly my plan. I was working on my thesis, and the program I was writing for it produced tons of log output I had to filter before I could make head or tail of it. When this turned out to be too complicated, I turned to colouring the matched lines instead, which was useful enough and allowed me to finish my thesis.

Afterwards, I tried several times to hack deep enough into Eclipse’s console classes to implement line filtering, but the result was never stable enough to be usable. So instead, I came up with the next best thing: Grep View. It’s basically the same thing, only in a separate view.

You can use Grep View instead of the regular console, or in addition to it as an overview of your console output. One important restriction is that Grep View cannot deal with changes of existing content in the console document – that is, if your console document changes text in the middle of the console output instead of at the end, Grep View will still add the new text at the end. Also, if Eclipse purges earlier console output because the console content becomes too long, Grep View ignores that. But since console output is usually linear (i.e. once a line of text is in there, it doesn’t change later on), and Grep Console’s content should normally be much shorter than the full console output (the point being to show only relevant lines from the full output), that should rarely be a problem.

So, how do you use it?

Using the Grep View

Simple enough. Let’s once again use the same demo program from the previous how-tos:

public static void main(String[] args)
{
  System.out.println("This is the first line.");
  System.out.println("Some more text...");
  System.out.println("This is the second line.");
  System.out.println("Some more text...");
  System.out.println("This is the third line.");
}

And again, we continue with the Grep Console settings from last time – get them here and load them in the Grep Console dialog in case you no longer have the howto configuration.

For starters, just run the demo program once again. The console output should look like it did last time:

Now, open the Grep View via Eclipse’s Window/Show view/Other menu:

This will show the Grep View, which so far is still empty:

As you can see, the Grep Console copies the Grep Console and scroll lock buttons from the regular console. There’s also a drop down button to choose which console the Grep View should show, and a button to lock the currently displayed console to whatever Eclipse’s own console view is currently showing – this is enabled by default. At this point, the Grep View therefore already shows the console containing our demo output.

Then why is it empty?

Simple: Because none of our expressions were configured to be filtered to the Grep View – and everything that’s not marked for filtering is not displayed in the Grep View.

But that’s easy to fix. We already have expressions to highlight the three “This is the…line” lines in our output. To enable them for Grep View filtering, just open the Grep Console dialog (via the button in the console view or the Grep View – doesn’t matter):

A look at the right icon column, which is used to configure, filtering, shows that for our current launch configuration, all filtering values are left at their defaults (shown by filter icons within brackets). And while all our expressions are enabled for filtering by default (a filled filter icon), the “How-to 1” folder which contains them is disabled for filtering (empty filter icon).

Clicking the filter icon next to the folder until it shows a filled filter enables filtering for the folder. But wait – now all our lines would be filtered to the Grep View, rendering the whole exercise pretty pointless. We only want the “This is the…line” lines filtered. So we simply disable filtering for all expressions in our folder except for the one that reads (\Qline\E):

Now click “Ok”, and the Grep View will show just what we ordered. Even better – it applies the same style highlighting we already configured for the regular console output:

In case you wonder why the background colour styles show up here although we disabled their expressions for filtering: The lines themselves show up of course because we enabled the “line” expression, which is enough to get them into Grep View. And once a line shows up, it’s styled with all expressions that are enabled for styling, even if they’re not enabled for filtering.

In other words: The filter settings determine what lines are shown in the Grep View. But those lines that do show up there use the same styling settings they also have in the standard console view.

Performance considerations

If your program produces a lot of log output with long lines at high speeds, Grep View can become a bottleneck. Testing regular expressions against long lines of text is an expensive operation (the longer the line, the slower it is). And while Eclipse is clever enough to only test those lines for styling expressions that are currently visible, the Grep View can’t be so picky: You want all your filtered lines to show up there, so all lines from the console have to be tested against all filtering expressions.

One way to deal with this is to shorten the lines. In fact, Grep Console already does that. The settings page in the Eclipse preferences defines the maximum line length used for testing style and filter expressions:

The numbers here define the number of characters after which a line of console output is cut before testing a regular expression against it. This doesn’t affect the lines that are actually shown in the console – lines longer than the values entered here will still show up in full length. But if the part matching a regular expression occurs after the cutoff point in the line, the expression will not match the shortened line and the highlighting or filtering will not be applied. Similarly, the “whole line” style will still be applied to the whole line, but capture groups after the cutoff point will not be used.

As style matching is the less expensive operation (being only applied to currently visible lines), the default character limit for style matching is higher than that for filtering. In case your regular expressions really need longer lines, you can increase the limits at the expense of performance. You can even disable line shortening altogether by setting these values to 0. Careful though, if your program outputs a lot of very long lines at a very high speed, too long filtering limits can effectively cause your entire Eclipse workbench to stop responding. If that happens, kill Eclipse, reduce the matching limits, and try again.

Likewise, if you feel Grep Console is slowing down your program, try reducing the values here. If that doesn’t help, make sure you’ve only enabled those expressions for filtering that you really need – it’s not the number of expressions that actually match that eats your performance, but the total number of all expressions tested against your lines, even if they don’t match.

Feedback

That’s it as far as these how-tos go. I should have covered all relevant features of Grep Console now. Remember, I’m open for all kinds of feedback, bug reports and suggestions. Just use the “Feedback” link in the Grep Console menu at the top of this page to get in touch. Thank you for using Grep Console!

Note: Grep Console 3.0 and 3.0.1 had a bug that caused some expressions to be created not quite as described here. This was fixed in version 3.0.2. If you still have an older version, please update before continuing with this how-to.

After the last two how-tos described the basics of Grep Console, it’s time for some advanced stuff. This time, I’ll be showing how to use capture groups to highlight only certain keywords in a line.

Once more, we’ll reuse the code from the previous examples for our demo program:

public static void main(String[] args)
{
  System.out.println("This is the first line.");
  System.out.println("Some more text...");
  System.out.println("This is the second line.");
  System.out.println("Some more text...");
  System.out.println("This is the third line.");
}

Also, here‘s the settings file containing the grep expressions created during the first two how-tos, in case you don’t have them on your system. Simply load the file in the Grep Console dialog.

We now want to add an expression to highlight only the single word “line”. The easiest way to do this is to run our program once, select the word “line” in the output and right-click to open the context menu, then select “Add expression” and the “How-to 1” folder.

Take a look at the grep expression this shows in the expression dialog:

As you can see, the text “line” is surrounded by parentheses. In regular expressions, they are used to denote a capture group. Capture groups can be used to select certain parts of lines matched by the regular expression as a whole. Remember that Grep Console automatically wraps every expressions with “.*” strings at the beginning and end. The full expression in this case therefore looks like this:

.*(\Qline\E).*

The “\Q…\E” part just means that any special characters between the “\Q” and the “\E” should not be treated as regular expression characters – we could write “.*” inside this part and it would only match a dot character followed by an asterisk, not any number of arbitrary characters as “.*” usually does in a regular expression. In this case, “line” is a simple enough string without any special implications, so we could just as well drop the “\Q” and “\E” without problems.

The whole expression therefore means: Match lines that contain any number of arbitrary characters followed by the string “line” and any number of additional arbitrary characters, and make “line” a capture group.

Grep Console recognises capture groups and allows you to assign a distinct style to each capture group. So far, we’ve only used the “whole line” group, which is present for every regular expression and applies, as the name says, to the entire matched line. Now that we have a capture group, the table below the expression field shows a second line, labelled “Group 1”. This corresponds to our capture group, and a style applied to this group will only be used on the part of the line described by the group.

Create a new style for the capture group by double clicking on the “Group 1” line and enable bold font in the style dialog. The preview should now look like this:

As you can see, every line containing the word “line” has that word highlighted in a bold font.

But what about the expressions we already created earlier to highlight those lines? Click “Ok” to close the dialog and take a look at what the console output looks like now:

Grep Console combines all styles matching a line. In this case, we’ve specified that three of our console lines should be styled with a background colour, and that the word “line” should be bold. The result is just that: Three lines with background colour and a bold font for the word “line”.

But you can also assign different styles to different parts of a line with a single expression. Open the Grep Console dialog and create a new expression using the following regular expression:

(Some)(.*)(text)

If you type this text into the expression field, you may notice the text sometimes turning red as you type. If the expression is shown in a red font, it means that the current expression is invalid. Hovering the mouse cursor over the text field will open a tool tip showing a description of the error, as provided by the Java API. For example, if you type an opening parenthesis without a matching closing one, the expression will be invalid.

The above expression contains three capture groups, and the table below the expression field will therefore have four entries: “Whole line”, “Group 1”, “Group 2” and “Group 3”. Create a new background colour style for the “whole line” group, assign the “bold” style we created above to groups 1 and 3, and assign the background colour style used for the second line (named “Line 2” in my example) to group 2.

The result should look like this:

As you can see, the second capture group spans all the text between the first (“Some”) and third (“text”) group, including white space. Capture group styles take precedence over styles applied to the whole line, so the middle group is shown with a blue background, while the rest of the line uses a pink one.

Capture groups are an excellent tool to highlight relevant parts of log output lines. For example, if your program regularly outputs a certain numeric value of interest, you can use a regular expression to identify all lines containing this value, and then use a capture group to apply a text style to only the numeric value. This allows you to easily find the relevant information in your log output.

In closing, here’s a look at the fully styled console output:

The next how-to will introduce the Grep View.