Tools of the Trade: AppCode, a breath of fresh air from the Xcode monoculture.

If you are a Mac or iOS developer for better or for worse there is no way around Xcode.

Xcode is free and full-featured, so why would you ever want to use anything else? This is the main reason why there are practically no other Mac OS X or iOS developer tools on the market today. There just isn’t any room for third parties for it to make economic sense to develop expensive developer tools.

The only other serious IDE for Mac OS X and iOS development is JetBrain’s AppCode and I’d recommend that every serious Apple developer should own a copy. While Xcode has evolved into a powerful and mostly stable tool, Apple has a lot of blindspots and Xcode is in many areas (at least) 15 years behind the top of crop. AppCode isn’t.

JetBrains is the powerhouse of Java development tools and they represent everything that Apple does not. Where Apple is closed, secretive and has a very paternalistic approach to its developer community, JetBrains is open, transparent, friendly and as cross-platform as it is possible to be.

The advantages for an Apple developer such as myself is that you get a peak at the world beyond Apple’s strictly enforced white room monoculture. Using AppCode is as much about growing as a developer as it is about efficiently developing software.

JetBrains offers IDEs that support nearly every language that is available and the more outrageously new and niche a language is, the more likely that JetBrains has a tool for it. This means that once you get used to the basic IDE concepts, you can take that expertise and use it for developing in other languages, on other platforms (Android, Windows, Web) and with other technology stacks.

I use WebStorm for my own website development, RubyMine for web app stuff and IntelliJ IDEA for learning functional programming in Scala. If I ever wanted to learn CoffeeScript, Dart or Haskell I know I’d be covered there too. On top of this JetBrains’ plug-in technology makes adding support for the latest and greatest open source technologies a breeze and JetBrains are very good at keeping an eye open for exiting new technologies. There’s a good chance that the first you hear about a new technology is by looking at JetBrains’ product release notes.

The AppCode IDE itself is very much in the mold of other Java development environments. The IDE can do everything and more, but it is also very busy and a long way from the pared-down minimalistic Apple aesthetic. It’s a nerdy power tool more than a philosophical statement.

JetBrains is rightly famous for their language parsing and refactoring acumen, so their IDEs are chock full of “intelligent” features. Not the kind of “intelligent” that makes everything harder, but the actual intelligent kind.

Navigating in AppCode is much more powerful than in Xcode. The gutter contains a myriad of options that will take you from method implementation to declarations and vice versa. You can also click and hold from class definitions to jump to super- and sub-classes, get in-line help and auto-fixing for commons problems. The as-you-type code analyzer finds potential problems and standard fixes, the code reformatting options are powerful and easily accessible. The intelligence extends to seamlessly into finding all places a piece of code is actually used rather than having to rely on text searches.

Best of all, however, AppCode can make changes to associated files without leaving the current file. The annoying roundtrip between implementation and header files that keeps interrupting your train of thought in Xcode can be wholly avoided. You write the implementation for a method and AppCode just offers you the ability to declare said method in the header with a single click without ever taking your eyes of the code you are busy writing.

Working in AppCode you constantly find yourself wondering why Apple can’t just do this. If it seems obvious, it’s in AppCode. Unfortunately this is rarely true for Xcode.

Refactoring is part and parcel of the AppCode experience and backed so far into the IDE that it becomes a nearly invisible part of your development. If you are used to refactoring in Xcode, you are likely to be non-plussed by AppCode’s refactoring support. Where Xcode makes a huge deal out of every refactoring: taking a snapshot, making you validate a thousand changes and more likely than not failing bang in the middle of the refactoring, AppCode just makes the changes with no fuss whatsoever. The first time I used the renaming refactoring in AppCode, I was wondering what I was doing wrong. I typed the new name into the red highlighted area and nothing happened! How do you terminate the editing? In fact, AppCode had already done the project-wide refactoring. Why make a fuss about it? Why could it fail? Why beach-ball for a few seconds? Why indeed?

AppCode enables you to work in a completely different manner to Xcode. Say you are into Test-Driven Development. Write the test cases first. When you instantiate your target class in the test class, AppCode will tell you that the class does not yet exist. A single click solves the problem by creating the class for you. As you write your tests, you can one-click to add method declarations and empty implementations. When you’ve finished with your test cases, there’ll be .m and .h files with complete stub implementations all without you ever leaving the test case implementation file.

Another big differences with Xcode is that where Apple knows everything best and either offers no customization or forces you to comply with their guidelines, JetBrains puts you in charge. Almost every aspect of the IDE is fully customizable: you can define your own coding style, which will cause AppCode to use your specific style to create stubs. You can even decide to reformat your code automatically before checking it into source control. You can (obviously) choose your own source code management system, add CocoaPods support, edit and preview HTML, CSS, Compass, TypeScript, JavaScript, files or add your own selection of plug-ins. In short, JetBrains is for grown-ups that like taking their own decisions.

Similarly, if you’ve ever felt the frustration of never being able to talk to anybody at Apple about Xcode, you will find the JetBrains support team a breath of fresh air. Something not working? Something not supported? Something you’d like to see added? Just drop them a line and an actual person will reply to you; better yet that person will be an approachable, open-minded fellow developer intent on helping you out. With JetBrains you’re the customer and you know best.

Seriously, just give it a shot. If only for a breath of fresh air.

The unbearable fragility of modern Mac OS X development

There I’ve done it again: I shipped a broken A Better Finder Rename release despite doubling down on build system verification, code signing requirements validation and gatekeeper acceptance checks, automation, quality assurance measures, etc.

Only in October, I had a similar issue. Luckily that time around it only took a few minutes to become aware of the problem and a few hours to ship a fix so very few users were affected. Right now I don’t know how many users were affected by the “botched” A Better Finder Rename 10.01 release.

This didn’t use to happen. Despite the fact that I did not spend nearly as much time ensuring that everything worked properly with the release management. Nor am I alone in this situation. Lots of big as well as small developers have recently shipped similarly compromised releases.

The situation on the Mac App Store is much, much worse. Nobody other than Apple knows how many Mac App Store customers were affected by the recent MAS certificate fiasco that had the distinction of making it all the way into the pages of Fortune magazine.

The truth is that Mac OS X development has become so very fragile.

The reasons for this are manifold and diverse but boil down to: too much changetoo little communicationtoo much complexity and finally too little change management and quality control at Apple.

The recent Mac App Store (MAS) fiasco that left many (1% of Mac App Store users? 100%? Nobody knows) users unable to use their apps purchased from the Mac App Store was down to Apple’s root certificate expiring. This was a planned event: certificates are used for digitally signing applications and they are only valid for a particular period of time, after which they need to be replaced with new certificates.

When the Mac App Store certificate expired, it was replaced with a new certificate but there were two problems. First, the now expired certificate was still cached by some of Apple’s servers: when Mac OS X opens an application it checks its signature, which in the end is guaranteed by Apple’s root certificate. Since this was no longer valid, Mac OS X refused to launch them and reported them as “broken”, leaving users and developers equally baffled. After far too long, Apple investigated the problem and emptied their caches which made the problem go away.

The second problem which was not solved by updating the caches, was due to Apple also replacing the certificate with a new, higher security version; of course without telling anybody. The new certificate could not be verified with the old version of OpenSSL that was used in the receipt checking code of many shipping apps.

When Apple created the Mac App Store, it provided a “receipt” that each application should check to see whether it has been properly bought on the Mac App Store. This is just a signed file that contains details about what was bought and when. Instead of doing the obvious thing, which would have been to provide developers with an API for checking the validity of the receipt against Apple’s own rules, they just publishing snippets of sample code so that each developer could “roll their own” verification code. Supposedly this was for added security (by not providing a single point of failure), but it seems more likely that they couldn’t be bothered to ship an API just for the Mac App Store.

This decision came back to haunt them, because most developers are not crypto experts and so had to rely on developer contributed code to check their app’s receipts. Once this worked properly, most developers wouldn’t dream of touching the code again.. which is how it came to pass that many, quite possibly a majority, of Mac App Store apps shipped with the same old receipt checking code in 2015 that they originally shipped with in 2010(?). This was fixed by Apple revoking the new style certificate and downgrading it to the old standard.

For once, I had been ahead of the curve and had recently updated all the receipt code in my applications (no small feat) and I have yet to hear from any customers who had problems.

Just before the Mac App Store fiasco, however, many non-MAS had also shipped with broken auto-update functionality.

Apple does not offer any auto-update facility for applications that are not on the Mac App Store, which lead to Andy Matuschak’s “Sparkle” framework becoming the de-facto standard for adding an auto-update features to Mac applications.

Driven by abuse of some HTTP communications in iOS apps, Apple decided that in iOS 9 it would by default opt all developers into using only (more secure) HTTPS connections within their applications. What is good for iOS 9 can’t be bad for Mac OS X 10.11 El Capitan, so Mac applications also got opted into this scheme.

Unfortunately, that broke Sparkle for applications which do not point to HTTPS “app casts” such as mine. I have long resisted installing my own HTTPS certificates because I was worried about messing up the expiry periods, etc.. apparently just the way that Apple did with the Mac App Store certificates.

Most developers will have been unaware of the change since Apple never announced it, but I had happened to see the WWDC conference videos that mentioned this in passing. Unfortunately, nothing “clicked” in my head when I heard this. My applications do not communicate with a server anywhere and I thus thought that this was not something I had to worry about. I had forgotten that Sparkle might use this internally.

Fortunately, I caught this at 6AM when I released A Better Finder Rename 10 final. I was just doing a normally completely redundant check through all the features of the program when I noticed that the new version failed when trying to check for updates. By 8AM, I had identified and fixed the problem so that very few people indeed could have been caught out by it. That was luck though.

The nefarious element here was that applications were opted in automatically and silently. Before 10.11 El Capitan was installed on your Mac, my applications updated just fine. Afterwards, they no longer did. Just because they were on El Capitan. Gee thanks!

Of course, this would not have happened if I hadn’t built A Better Finder Rename 10 with the Mac OS X 10.11 SDK (Software Development Kit) at the last moment.

It is somewhat crazy for a developer to change the SDK that s/he builds a brand-new version of their software against in the middle of the beta phase. Changing the SDK always introduces errors because the entire environment in which the code executes is changed. This may bring out bugs that were already present; things that should never have worked, but worked just because the API happened not to trigger the bug. It also introduces bugs that are just part of the new SDK and that you now have to work around. Changing SDKs makes existing programs fragile.

I’m very conservative when it comes to changing SDKs because I’m well aware of the risks. That’s why I’ve been building my code against older SDKs for the past 15 years. A Better Finder Rename 10 was built against the Mac OS X 10.7 SDK which is forwards-compatible with newer versions of Mac OS X.

The main reason for doing so, is that I wanted to be certain that I didn’t accidentally break A Better Finder Rename on older systems, which brings us to the next problem with Mac OS X development.

Xcode lets you specify a “deployment target”, for instance 10.7, while building with a newer SDK. This is the recommended way of developing on Mac OS X and keeping backwards compatibility. Xcode will, however, happily let you use APIs that are not compatible with your deployment target and thereby ensure that your application will crash on anything other than the latest Mac OS X.

In fact, Xcode encourages you to use the latest features that are not backwards compatible and will rewrite your code for you if you let it, so that it will crash. It will give you “deprecation warnings” for any API usage that is not in the latest SDK and resolving those warnings is likely to break backwards compatibly as well. Of course, you won’t know this until you run it on the old Mac OS X version.

Now which developer can afford to keep testing rigs with 10.7, 10.8, 10.9 and 10.10? Never mind spend the time re-testing every change on multiple platforms for each change?

Thus I happily built with the 10.7 SDK. Apple did not make this easy by not shipping the old SDKs with Xcode, but you could manually install them and they would work just fine.

Imagine my surprise after installing Xcode 7 and finding out that this no longer worked. The only workable solution was to build against the 10.11 SDK, so jumping forwards not one but 4 SDK versions. A bunch of code wouldn’t compile any longer because the libraries were gone. Luckily the receipt checking code was amongst those, so it got modernised just in time to avoid the Mac App Store receipt fiasco.

Nonetheless, now my entire code base had become fragile and largely un-tested between the last beta release and the final shipping product. Nightmare!

On top of that was it still even 10.7 compatible? or indeed 10.10 compatible? Just quickly running it on older systems wouldn’t provide more than a little additional confidence since it’s impossible to go through every code path of a complex product.

After installing virtual machines to test on, I still couldn’t be 100% certain. The solution came in the form of deploymate, a now truly essential developer tool which does what Xcode can’t do: check that API usage is compatible with the deployment target.

I have since spent many weeks trying to ensure that I won’t run into the same problems again by adding (additional) automated verification processes to my build system. My build system now runs the built product through SDK compatibility checking courtesy of deploymate, code signing validation and gatekeeper verifications on each build. I’m still working though deprecation warnings and the like and my code base will soon be bullet proofed at least until the next forced changes arrive.

You’d think that this was a long enough list of problems for one year, but this still does not account for Apple also changing the code signing rules (once again) earlier in the year (in a point update of 10.10 no less). This time it affected how resources and frameworks are signed. So applications that were signed correctly for years, now suddenly became incorrectly signed and Mac OS X would refuse to launch them because they were “broken”.

All this points to the underlying issues with the current spade of fragility of Mac applications: Apple keeps changing the status quo and neither it, nor developers have any chance of keeping up.

Apple’s own applications are full of bugs now. None more so than Xcode, which is both the lynch pin of all Mac OS X, iOS, watchOS and tvOS development and no doubt Apple most fragile app offering. Xcode is in beta at least 6 months a year and never really stabilises in between. Each new version has new “improvements” to code signing, app store uploading, verification code, etc.. and each new version breaks existing code and introduces its very own new bugs and crashes. From one day to the next, you don’t know as a developer whether your code works or not. Existing code that had worked fine on Friday evening, no longer works on Monday morning. Worse, chances are that you are not hunting for your own bugs, but those in your development tools, the operating system or Apple supplied SDKs.

All this is driven by the one-release-a-year schedule that Apple has imposed on itself. This leaves all of Apple’s software in various stages of brokenness. When Apple’s own staff cannot deal with this constantly shifting environment, how are third party developers supposed to?

Case in point: Apple’s own apps are not all iOS 9 compatible yet. Many don’t support the iPad Pro’s new native resolution yet. Some have gained Apple Watch extensions, but most haven’t.

Reliability is a property of a system that is changed slowly and deliberately and where all constitute parts are themselves reliable. The Mac and all other Apple platforms are currently undergoing the worst dip in reliability since Mac OS X was introduced.

Apple is pushing out half-baked annual releases of all its software products, as well as apparently completely unmanaged changes to policies, external rules and cloud services at an ever more frenetic pace.

These could be written off as temporary “growing pains”, but the big question is: Do all these annual updates equate to real progress?

When I switch on my Mac today, I use it for much the same things that I used it for 10 years ago. A lot has changed. Cumulatively Mac OS X 10.11 El Capitan is somewhat better than 10.6 Snow Leopard.. and yet if you discount cosmetic changes and new hardware, nothing much has changed. Certainly nothing much has actually improved.

I can’t help thinking that if we had had 2 or possibly 3 Mac OS X updates instead of 5 over those last 5 years, we’d be in a much better shape now. Apple and developers would have time to provide user benefits and rock solid reliability rather than just endlessly chasing their own tail.

The beauty of the Mac used to be that it just worked. I want to get back to that.

Some thoughts about Optionals in Swift

Like many Mac and iOS developers, I haven’t yet made the jump from Objective-C to Swift, nor will I make it in the near term.

Just like most other developers, I am slowly putting a little bit of Swift code into new projects to get a feel for the language and I’m experimenting with its new features to be ready when it becomes a more practical proposition.

At the moment, I’m trying to come to grips with “optionals“, a construct that I have never seen in any language I’ve used before.

Essentially, “optionals” are shoeboxes that may or may not contain a value. You can find out whether they contain a value and if so, you can get the value out by “unwrapping it” and if it contains a mutable type, you can change it.

This replaces the C-style pointers that may either point to the data belonging to an object (in the largest sense of the word) or have a “nil” or “NULL” value that signifies “no object”.

Optionals are a clear improvement on null pointers or nil types in at least two significant ways:

  1. Optionals eliminate a whole class of common bugs by forcing developers to think about “what happens if this method does not return a valid object/ value”
  2. Optionals help make Swift code faster by making it easier and safer to optimize code; optimizers do not have to worry about edge cases where the value may be empty.

The price for those two advantages seems very steep, however.

First, not dealing with potentially invalid objects being returned prevents the code from being compiled in the first place. You cannot not deal with the problem.

Secondly, optionals require developers to learn a host of new concepts. This may be a good thing in the long term, but it will slow adoption as you can’t just dive into the language without understanding optionals and you have little hope of ever having come across them before (unless you are beta testing Rust that is).

Using optionals is also really, really cumbersome. Unwrapping each and every optional makes even trivial programs balloon to many times their normal size. This is why Apple has thoughtfully added a lot of syntactic sugar to make the medicine go down that much more easily.

This syntactic sugar (chaining, etc.) makes your code smaller and cleaner and you can almost forget that it is there at all when you read it. Unfortunately that isn’t the case when you write the code and it does also add a whole layer of complexity making Swift harder to learn and understand.

Optionals also do not map onto anything in Objective-C and as there are no native libraries yet, “implicitly unwrapped optionals” abound and feel very much like a backwards-compatibility hack and a solution to a problem you wouldn’t have without optionals.

So there is a price to be paid for beauty, performance and safety.

That needn’t be a show stopper. You get nothing for free in life and the result may be worth the effort.

Unfortunately, when I’m actually using Swift, I find that I don’t seem to be getting much value out of optionals.

Firstly, there’s the fact that now instead of trying to express what I want to do, I find myself wondering about whether I need to unwrap my optionals? and what might be the best syntax for expressing the unwrapping? where should it be done? now or later? Why does this code not compile?

Admittedly with a little bit of practice, this will probably largely go away. There will always be a  cognitive overhead associated with it, but it will that overhead is likely to get smaller with time. I certainly won’t miss having to check the location member of the return value of rangeOfString: for NSNotFound.

The real killer for me is, however, that I now find all those places in my code where I need to think about what to do when the optional is empty .. and I find that there’s not much to be done.

What can you do when you are loading an image from your app bundle and it is not there? There only seem to be two options:

  1. raise an exception, crash or display an alert that the user can’t do anything about
  2. do nothing

Crashing is great, because now there’s a crash report being generated and I know that something has gone wrong in the production version and I can work out what the cause is and correct it. Crashing early is best practice. It certainly beats crashing later.

Doing nothing might be okay in many situations, because it might not be so bad. It is in fact what Objective-C does most of the time in this situation. It just ignores the problem, the application keeps running (but not necessarily working) or it simply crashes somewhere down the line.

Herein lies the problem with optionals. They force you to deal with potential problems that you can’t do anything intelligent about. Worse, because of the absence of an exception mechanism you need to deal with exceptions in situ, polluting your beautiful code with non-sensical error handling code.

So, what I’m wondering is: “Is there any value in explicitly dealing with problems that you can’t do anything about?”