Fun with regular expressions

Regular expressions are amongst the most powerful text manipulation features around. Most dynamic websites that you see around the web are based on programming languages that incorporate regular expression support, such as PHP, Perl, etc. Harvesting this power for file renaming tasks, however, is not so very straight forward. A Better Finder Rename has supported regular expressions for a couple of years now. When the feature was first introduced, I tried to help users figure out what is going on by providing a preview of the various substitution groups of the first file to be renamed. This was miles better than no special preview at all, but fell short of what I really had in mind. Version 7.0 was supposed to have the “new improved” regular expression preview, but the scope of the release just kept growing; mostly due to the mountains of great feedback that I received from my private beta testing group (thanks guys!). Still version 7.0 was a great step forward, even with one or two planned features not making it. Fans of the program will know that I add a new feature pretty much every month and have done so for the past 10 years, so the day had to come when the missing feature would finally make it. Mission accomplished with version 7.3.5 out today. So why are regular expressions so hard? and why are they so powerful? The answer to both is the same: it’s programming with text. Programming is hard; programming is powerful. Regular expressions are a “pattern manipulation language” and their syntax is both super “tight” and profoundly cryptic. Just the way that Unix geeks love things to be. Manipulating text with “reg ex” is a matter of first identifying groups of letters (or “symbols”) in the existing text (i.e. the current file name) and then rearranging the groups and perhaps adding letters (“symbols”) to the “output”. Let’s see a brief example: regx_preview_thumb1.gif As you can see the current name is “hello world” and we simply want to swap both words. First we need to identify the two words. We do this with the pattern “(.*) (.*)”. What does this mean? Well a “.” is a placeholder for any symbol. A ” ” (space) is a placeholder for a space and an “*” means that there may be 0 or more occurences of the last symbol. So “.* .*” is a pattern that matches any text that has a space somewhere. So for instance, “the cat”, “the mouse” and “the cat and the mouse” all match the pattern “.* .*” because they have a space somewhere. “the_mouse” does not match because it does not have a space. A Better Finder Rename will simply do nothing for a current file name that does not match the pattern. In other words, “the_mouse” file will be left untouched. In the screenshot you can see that our pattern is not just “.* .*”, but “(.*) (.*)”. The brackets match nothing but simply enclose a substitution group. These substitution groups can be used in the substitution expression to refer back to what was matched. Each substitution group has a “name”: \1 is the first substitution group, \2 the second, etc. A Better Finder Rename supports up to 8 substitution groups. In our example, we split “hello world” into two substitution groups: \1 which is “hello” and \2 which is “world”. In the substitution field we have put “\2 \1”, which translates into “the contents of the second substitution group, followed by a space, followed by the first substitution group”. In other words, “swap the first and the second words” of the file name. This is of course barely touching the surface of the regular expressions. Things become more interesting when you start “programming” in earnest. Say we want to swap the position of the numbers at the end of some image files in a “clever” way: regex_full_preview_2.gif We are only interested in the numbers, so we say “match all uppercase or lowercase letters” at the beginning of the name up to the first number and put all the numbers into the substitution group \1″. We then use the numbers substitution group in our substitution expression. Voila. Obviously, it isn’t really possible to cover regular expressions in detail in a blog entry. Whole books have been written on the subject. The manual page for the feature has some more details on the syntax, the built-in support and a selection of further reading materials, including some useful books. Most of you will go “oh no this is far too complicated for me” at this point. That’s ok, this is an advanced feature for advanced users: there are plenty of easy to use features in the program and you can achieve most frequent file renaming operations without regular expressions. For the “advanced users” amongst you, however, this will alert you to the presence of the feature and might motivate you to learn a little more..

Poll: Should the “Apply” button become the default button in A Better Finder Rename?

Seasoned A Better Finder Rename users will probably remember the good old days when A Better Finder Rename was “only” a context menu that was neatly hidden away in the Finder contextual menu.

With version 7.0, however, A Better Finder Rename became a full blown “stand-alone” application that may also be launchied via the context menu. This meant that some user interface rework had to be done. Throughout this, I tried to disrupt the working habits of existing users as little as possible.

New users, who have never known the “context menu only” A Better Finder Rename find some of its behaviour a little odd however.

Chief amongst these annoyances is the behaviour of the default “Rename” button. This, most new users feel, should simply peform the rename and stay open. I tend to agree. It’s only logical..

The problem is that when A Better Finder Rename was only a context menu item, it made more sense that it would be behave like the “Get Info” dialog: You make your changes and then the dialog closes, the Finder window shows you the new file names. Neat.

This is the reason why even the latest version 7.3 (for new users “inexplicably”) quits just after peforming the rename.

With the new version I have added an “Apply” button, which performs the rename, empties the preview list and stays open.

abfrx73.gif

Now it is represented to me that this should be the default behaviour, ie. I should get rid of the “Rename” button behaviour and make the “Apply” button’s behaviour the default. Hitting the return key will apply the changes, but not quit the application.

This is of course the request of a user who has not spent the last decade hitting the return key to perform a one key stroke “rename and quit”. My highly scientific “one user poll” shows that 100 percent of existing users don’t want that.

How is a software developer to know which option to choose? Why, he could just ask his users..

So please cast your vote below:

I will probably only change the behaviour if there is a strong (two thirds?) vote in favour of the change..

TheMacNurse competition

What makes the “world of mac” different from the Windoze world is largely the spirit of community and it’s fascinating that after 30 years of Apple, the community is probably as vibrant as it is has ever been.

There are community sites, news sites, rumor sites (this is no doubt the Mac speciality) and all kinds of other enthusiast endeavours springing up all over the net.

TheMacNurse, is one that has piqued my particular interest. They offer free online support for all things Mac. That can’t be right “free”?

Well it seems it is, and their volunteers will help you out at no cost.

This kind of inititiative is just too great to stand idly by and do nothing. As I can’t honestly say that I want to join their volunteers and provide free Mac support in addition to the technical support I provide for our own products, I’ve taken the easy way out and chosen to give away two copies each of:

So go ahead and visit their competition page at: TheMacNurse.

All the best for the future guys!

Changes ahead for the A Better Finder series in 2006

2005 was a year of transition for the A Better Finder series of tools. Most of the year was spent migrating the 60,000+ lines of code of A Better Finder Rename 6 to the brave new world of Objective-C and Cocoa.

I took the opportunity to add many long-requested features, such a detachable preview window, multiple rename steps, etc.. One of the most important changes was to introduce drag & drop installation. The kind people at MindVision have provided me with their InstallerVise installer maker for ten years, but the product was beginning to show its age and its Mac OS 9 legacy.

Enter the 2005 drag & drop style installer. Today all you need to do to install A Better Finder Rename 7 is to take its icon from the disk volume and drop it where you want it. Double-click to start and you're finished. For multi-user installations, simply place the program in the Applications folder and let every user decide which optional features they want to install. This new drag & drop installation is now making its way across the entire product line:

already work on the same principle.

This leaves A Better Finder Select and A Better Finder Creators & Types. Once upon a time, both of these products covered a niche left open by the Mac OS 9 Finder. Today both of them have somewhat lost their raison d'être.

"A Better Finder Creators & Types" allows die-hard Mac OS 9 fans to continue using creator and type codes to associate documents to applications, but this approach, while still supported under Mac OS X, is no longer the recommended way of doing things and does not work with newer applications.

A Better Finder Select allows you to filter out certain files before passing them to other A Better Finder products or it allows you to select them in the Finder; it's functionality is partially covered in the Finder and is at its most useful as a front-end to the other products in the A Better Finder Series.

Is it still really useful to keep them as separate applications? I don't think so. That's why in 2006:

  • A Better Finder Creator & Types' features will be integrated into the new A Better Finder Attributes 4
  • A Better Finder Select's filtering features will be integrated into the preview window of both Attributes 4 and Rename 7
  • A Better Finder Select's ability to pre-select files in the Finder will be integrated into Attributes 4

If you disagree with these arrangements, please post a comment or contact me via email. It is not too late yet 🙂

The advantages I see for you, the user, is that you will have less application clutter, less installation, a smaller download and last but not least will be able to filter out files in the preview window.

Obviously, with the end-of-line of Select and Creators & Types, I'll be offering free cross-grades to the owners of these "late" products.

A Better Finder Rename 7.2.5 Performance Beta

There can be little doubt that version 7 of A Better Finder Rename was a giant step ahead for the product in almost every respect.

The major area where that was perhaps not the case was in performance terms. As long as you are renaming only a couple of hundred files, performance is more than adequate, but for people dealing with many thousands of files the performance of version 7 was a step backwards. This came as a late and unpleasant surprise, because version 7's code base is just so much cleaner and more streamlined than that of version 6, which was beginning to show its age.

This is why I have taken some time out from developing new features to get to the bottom of this performance problem. This investigation has detected a couple of facts:

  • sorting is a real killer, especially when mp3 or exif tags are involved
  • feedback is just as important as raw speed
  • there is no way around the fact that doing the same thing one thousand times takes one thousand times longer than only doing it once

In this first iteration of getting a faster, nimbler renamer, I have concentrated on these points: 

  1. the program now only does the bare minimum processing while you're honing your settings. This makes for a fast and responsive preview even with tens of thousands of files. The preview only displays the first 250 items (that's around 10-15 screenfulls), enough to give you a real good impression of what the end result is going to be, not enough to bring the preview to a grinding halt. The heavy duty processing starts only once you hit the "OK" button.
  2. everything that can possibly be cached is cached. EXIF dates and the like are only read once and the system remembers them without re-reading them every time. This can result in a factor 10-100 performance improvement on large sorts.
  3. provide feedback on what's happening. Previous versions just went off to do their thing, leaving you to wonder what, if anything was going on. The new version provides a progress dialog that shows you how much work is actually going on behind the scenes and how much longer is going to be required to do it.
  4. I have found and eliminated a couple of bottlenecks in the program that result in improved overall performance

As a result of those changes, version 7.2.5b1 feels and behaves completely differently when confronted with huge renames. Yes, doing 2000 files will still take twice as long as renaming 1000, but the preview remains responsive and it's fun to see your Mac race through ten thousand name computations, sorting, validation, etc.

The downside of these improvements is that I've had to make fairly hefty changes to the structure of the program, including multi-threading its execution. Multi-threading, in particular, makes the complexity of a program explode because it causes all kinds of potential error conditions: deadlocks, racing conditions, etc.

This is why I've decided to release a beta version first to validate that my changes haven't broken anything that used to work just fine.

You can download the beta from the product download page . I trust you will find it a big improvement with large file sets.

Please do report any problems you may find; it's tempting to think "somebody else will or has already alerted the developer", but all too often nobody does. I can only fix bugs I know about and the best way of getting rid of them is to report them directly to me at: reiff@publicspace.net

Ergonomic Tip #2: Take regular breaks

After the first painfully obvious part of this column "Don't bang the keys!" comes the second, no-less obvious tip.

Taking no rest breaks may make you look like you're overworked and underpaid, but it will also make you tenser, more tired, prone to error and ultimately less productive.

The static posture in which we sit and type all day in front of our computers, the long hours of staring at a screen a few inches away from our eyes, the tens of thousands of keystrokes and the endless miles of mouse travel that we accumulate each day is sure to leave some trace at the end of each work day.

Just getting up for a few minutes allows our muscles to relax out of their cramped-up position, allows our eyes the luxury of looking at something else for a while and it gives a welcome break to our hands and wrists.

The notorious "They" recommend to take a five to ten minute break every twenty to thirty minutes of screen-based work.

I know what you're going to say though: I hate being interrupted when I'm in the middle of something. I lose focus and concentration when I get up. etc.

We all know the feeling of being "in the zone". We are in the middle of doing something complicated and after hours of unproductive screen gazing, we suddenly know exactly what needs to be done and how. And we are doing it. And we are enjoying doing it. And we are not going to stop until it's done.

This phenomenon is known to psychologists as "flow". When we experience "flow", we are working towards a clear goal, with optimal concentration, total efficiency and we may even experience a temporary "loss of self awareness". Should you stop because it might give you a headache in the evening? Probably not.

Most of our "work time", however, is spent in a much less ecstatic state of total productivity and this is particularly true for "knowledge workers". We spend a lot of time doing before thinking, and then going back and re-doing what we've already done after having properly thought about it. There is even a word for it: iteration.

Getting up and doing something else for a few minutes will help us clear our mind and let our subconscious get on with the job of sorting out what we should probably have done in the first place. How often have you finally figured out a complicated problem in the car on the way home, or in the shower in the morning, or when talking about the weekend's sport results with John from Engineering at the water cooler?

You can't force insight by sitting around feeling busy. You can work more that way, but you'll probably end up achieving less and the quality of your work will probably poorer for it. The point is that taking breaks doesn't make you less productive, it makes you healthier, less prone to error and ultimately happier at work and after it.

Rest break timers, such as our MacBreakZ or ergonomix products will remind you (sometimes perhaps too forcefully) to take a rest break. This can be annoying when you're in full flow and you will be tempted to just turn the damn thing off. Don't. Just ignore the dialog and take the break when you're ready.

When you're not in an ecstatic flow state, however, try to get up and take a walk. You'll feel better for it and trust me, you won't get less done.

MacBreakZ 4: Looking for alpha testers

It has finally happened: We have time to develop version 4 of MacBreakZ , our Personal Ergonomic Assistant.

MacBreakZ is what is commonly known as an "Ergonomic Rest Break Timer".

It is common knowledge that spending long, uninterrupted periods of time in front of your computer is the cause of many of our modern day occupational health disorders:

  • tension in neck, shoulders, upper and lower back
  • headaches
  • tendinitis of the fingers, hands, wrists and elbows
  • carpal tunnel syndrome
  • writers' cramp
  • and many other cumulative trauma diseases, occupational overuse syndromes and repetitive strain injuries

MacBreakZ combats these problems in a number of ways:

  1. reminds you to take breaks based on your health profile and your actual computer usage
  2. provides ergonomic advice and tips
  3. demonstrates and times stretching exercises that you can do in front of your computer

This type of software requires a high degree of adaptability and a particular sensitivity to the social factor influencing your work.

If you work in a home office and you are already suffering from one or more of the problems mentioned above, you will be looking for something that can help you recover and you won't care about much else. If you are, however, only looking to prevent injury and perhaps want to go home without a headache every day, you may be much more sensitive to what your co-workers think. You will not want to look silly doing strange stretches in an open plan office and you might be afraid that using a rest break timer might make you look less efficient or "injured".

Many of these things depend on your work setting and environment, the presence or absence of injury and your own mind set.

This is why, it is important for me as a product designer to be in touch with you, the end users of the product.

The development of A Better Finder Rename 7 was the first time, I developed a new product version in tight cooperation with a small number of alpha testers. This experience resulted in an end product that was far superior to what I could have achieved without the active help of this dedicated group of people.

This is why I want to attempt to replicate this approach with MacBreakZ 4.

I am looking for people who have time and effort to expend, who are interested in technology and have the desire to contribute to this project. Being accustomed to using MacBreakZ , ergonomix or any other break timer program will be an advantage.

If you feel like you might want to take part in shaping this new product and following it through its various development stages, please drop me a line at reiff@publicspace.net

Version 2: Even fewer features..

A Better Finder Launcher 2
It is not every day that a software developer removes features to make a better product, surely “more = better”. Yet, that is exactly what I have done with A Better Finder Launcher 2. Sometimes, I think, “less = better”.

Unlike all my other Mac software, my little launcher utility was written entirely for myself and it was never intended as a commercial product. It was born out of a desire to have a simple and clean little utility that would launch my frequently used programs and documents with just a few keystrokes.

Sure there were lots of great launchers out there already. Things such as LaunchBar (the blue print for Apple’s “revolutionary” SpotLight) and Quicksilver, but both those applications did such a lot more than what I wanted.

I didn’t want a program that manages my bookmarks, does a full-index search of my PDF files, is also a pasteboard and a coffee maker. Don’t get me wrong. They are great apps, they may be just what you are looking for, but they aren’t for me..

After tinkering around with my fun little utility for a while, I decided to share my little launcher with the growing community of A Better Finder users and it was released on an (unsuspecting) public as “A Better Finder Launcher”. It became a de-facto free addition to the A Better Finder Series package deal and has gone through ten point releases in its short life.

Version 2 brings the utility in line with A Better Finder Rename 7 and A Better Finder Attributes 4 in making the program both more of a stand-alone application and an easy drag & drop installation.

The adaptive ranking algorithm, however, is hands down my favorite new feature. It learns what your favorite applications and documents are and ranks them higher than less-frequently used items. This algorithm uses a combination of heuristics (e.g. “applications are more important than documents”), recency (e.g. “this program was last launched 5 minutes ago and is thus a better choice than that other program that was launched 3 weeks ago) and frequency (e.g. “Safari was launcher 3000 times, Firefox was only launcher 3 times. I think I’ll put my money on Safari”) to make guesses about which item is the best match. The greatest thing about it is that it works uncannily well!

I have enjoyed using version 2 for the past 2 months. I hope you’ll like it as much as me 🙂