Programming

Some things I wish I had know before starting to automate Mac Developer ID Notarization

It’s Day 5 of Notarization Week and it’s time to wrap up and write down my experiences.

Notarization itself is not incredibly difficult. You can learn the basics by watching the 40 minutes talk from WWDC 2019. Unlike sandboxing, notarization should not have any detrimental effects for most Mac apps.

As always the real trouble starts when you are trying to inject Notarization into the tangled web of modern Mac software development: entitlements, certificates, automated Xcode build chains, build settings, etc..

First you need to adopt the “Hardened Runtime” for your application. For the two apps that I tested with, this was simply a matter of switching it on in the “Capabilities” tab of your target. By default, all the hardened runtime features are switched on and I was able to leave them all on without any problem.

The first gotcha is that you can’t really test your application’s compatibility with the hardened runtime in Xcode, because it will run in debug mode. Since the hardened runtime would not allow inspection of your code, the default “CODE_SIGN_INJECT_BASE_ENTITLEMENTS=YES” build setting   will inject the “com.apple.security.get-task-allow” entitlement into the debug version of your build product. This is a “normal” entitlement, just like those used for sandboxing.. and no the sandbox does not need to be turned on for notarization to work (sigh of relief).

Another gotcha is that your app will not be notarized as long as this entitlement is switched on, so we need to turn it off for the release build. This should not be a worry, but you will probably spend many frustrating hours chasing down this very problem nonetheless..

The next thing on the compliance list is that secure timestamps for codesign need to be turned on. Many developers have a “–timestamp=none” flag somewhere in their build settings.. because the Apple timestamp servers are slow and often down (at least here in Luxembourg) and you can no longer build a release without an internet connection. So if you have a build server without internet connection.. that is about to change. To make doubly sure, you should probably add “OTHER_CODE_SIGN_FLAGS=’$(inherited)  –timestamp” to your build settings.

In this context, it would have saved me a lot of time if I had known how to find out whether a product has in fact been signed with a secure timestamp. Executing “codesign –verify –deep –strict –verbose=4 –display  -r- /path/to/my/product” will display loads of things. If there is a line with “Signed Time” among it, that means that you did not sign with a secure timestamp. If you have a line with “Timestamp” in it, it means you do have a secure timestamp. It’s another brilliant example of how an Apple engineer’s language choice can cost tens of thousands of lost developer hours. “Signed Time (insecure)” would have been a great help.

In a similar vein, “codesign -d –entitlements :- /path/to/my/product” displays all the entitlements for the product and will reveal the dreaded “com.apple.security.get-task-allow” entitlement if it is still present.

Once you have a build product, you can send it to Apple for notarization with the “xcrun altool –notarize-app -f /my/archive –primary-bundle-id my.primary.bundle.id -u username -p apps-peci-fic-pass-word”.

This is where things get a little weird. You can send either a disk image or a zip archive, but not an application bundle. I distribute my software as disk images and my software updates as zips. If you send a zip file, make sure that you use the “ditto” tool as instructed by Apple, so that you don’t run into problems with extended attributes. You need to supply your username (email address) and a password. You can generate an application specific password and that worked fine for me straight away.

The command line will upload the archive and then return a “request-id” which is a UUID that you can use to look up the state of the notarization. This is not a real-time, synchronous affair. It was fairly quick when I used it, taking usually only a few minutes, but it is obviously a challenging problem for automation. You could write a script that extracts the request-id and then polls the Apple servers for its status before continuing, but realistically you probably want to have a two or three stage build process now.

I subdivided my own build process from one into three phases: build, request notarization and a combined stapling and verification phase.

Which brings us to stapling, which is the fun and easy part. You just type “xcrun stapler staple my.dmg” or “xcrun stapler staple my.app” and that’s that.

One thing to note is that the entire notarization process is completely free of build and version numbers, which is so wonderful. If only app review worked this way! There is no mention on how it works; it could be that Apple uses the entire archive as a hash code or that they create a hash of your upload. In any event, there is zero problem building a thousand different versions of your program and getting them all notarized.

The second thing to notice is that you can either staple app bundles or disk images, but not zip archives. Not sure which is weirder, but it kind of makes sense. In practical terms, this means that you can staple your notarization receipt to a dmg without having to open it, which is super easy. If I have understood this correctly, this means that both the dmg and the app are stapled and will open without any funny user warnings. Not being able to staple zip files, however, complicates things somewhat, because you now have to zip the app bundle to notarize it, staple the original unzipped app bundle and then re-zip it.

So far so good. Now enter the much dreaded Sparkle.framework, the foundation of all automated software updates across the Developer ID world, maintained by a clever, intrepid group of volunteers that deserve our eternal gratitude.. and the bane of my developer life.

For most of my products, Sparkle is the only framework that I bundle, so I blame it for the entire dreaded complexity and wasted time of framework signing.. which is a lot of blame. Signing frameworks is hell.. or used to be hell.. and now is hell again.

I don’t use Carthage or other “download stuff from all over the internet written by who knows who, run buggy build and install scripts and then ship the whole lot with my product” build systems. I actually just place a binary copy of the framework into the /Library/Frameworks/ folder and work with that. If you are using one of those build systems, you probably will have different problems.

The current (as of 26/July/2019) binary distribution of Sparkle is neither signed, nor built with the hardened runtime, so is unusable for notarizated apps. Downloading the source as a zip archive leaves out crucial files. So I did a “git clone –recursive https://github.com/sparkle-project/Sparkle” to get what I assume must be the master branch version (I have some deeply strange git expertise that overlaps with nobody else’s).

Building it with “make release”, despite affirmations to the contrary, did not result in a hardened version. One of worst things (I’m pretty sure it’s unavoidable and I’m not dissing its developers at all, but it is still absolutely dreadful) about Sparkle is that it includes two executables as well as the framework. Autoupdate.app and fileop always cause incredible signing headaches. The default option of just ticking the “Sign upon copy” option in Xcode, won’t sign these properly and you inevitably end up with gatekeeper problems.. even though it had just gone through a phase of actually working.. but no more.

I’m sure that at the heart of all my signing problems is a lack of understanding, aka ignorance. The thing is that I’m a Mac developer, not a cryptography geek. Knowing just enough to get by in the context of cryptography means knowing quite a lot about quite a few things, followed by endless trial and error that eventually ends for unknowable reasons.

After a very long time, I finally got a Sparkle build that I could use by opening the project in Xcode, adding the “OTHER_CODE_SIGN_FLAGS=’$(inherited)  –timestamp”, “CODE_SIGN_INJECT_BASE_ENTITLEMENTS=YES” to every relevant target and manually adding my Developer ID signing identity to all targets. I have no idea why this was necessary; as far as I understand the framework does not need to be signed at all, and will in any event be re-signed when it is copied into my app, but it would not work without this. Perhaps the entitlements only get added during signing?

I then spent most of a day chasing down the origin of the “com.apple.security.get-task-allow” entitlement on the “fileop” executable that steadfastly refused to go away, despite having no debug build and having plastered the “CODE_SIGN_INJECT_BASE_ENTITLEMENTS=YES” build settings everywhere throughout the Sparkle project. Around 11PM, I decided to just delete Xcode’s “Derived” folder (what else was there left?).. and that promptly solved the problem.

With the Sparkle problems solved, the rest was fairly straightforward.

All told I’ve spent an entire week on learning about Notarization and integrating it into my build system.  It’s not badly designed. In fact it works fairly well and I would even go as far as calling some of the design decisions enlightened. It is certainly a lot better thought through than either App Review or the Sandbox.

Unfortunately, it adds yet more complexity to an already incredibly complex development environment. Today’s apps are not much more complex than those from the 1990s. Phone apps are mostly much less complex. It should be easier to develop for the Mac today than it was back in the 1990s. Yet nearly 30 years of development tool, framework and API progress has yielded a development context that is no more productive and far more complex. Notarization adds yet another layer of complexity and yet another time sink for Mac developers.

There are some positives: Apple can now revoke specific builds of an app, rather than just turning off all apps from the same developer id. The hardened runtime gives the developer the possibility of shielding his/her software from malicious modification, but allows him/her to decide which “holes” need to be blasted into runtime for the program to continue working. Actually scanning apps for malware adds peace of mind when you release a program into the world.

In an ideal world, Apple would turn around and ditch its Mac App Store sandbox requirement. It could even offer notarization as a way to side-load software on the iPad. After all, notarization gives it the tools to prevent malware from being published and to switch off on every single Mac in the world should it get through anyway.

As a long time Mac developer (since 1994), however, I can’t help thinking though that the security people at Apple would have done better ironing out the bugs and limitations of the sandbox to get it work properly and be less of a nuisance, rather than adding yet another security approach.

If early reports about Catalina are to be believed, it looks like there are so many people working on Mac security that they have to roll out new security features at each release, whether they are a net benefit to users or not. Perhaps, these people could be tasked with making macOS great again instead?

Tools of the Trade: AppCode, a breath of fresh air from the Xcode monoculture.

If you are a Mac or iOS developer for better or for worse there is no way around Xcode.

Xcode is free and full-featured, so why would you ever want to use anything else? This is the main reason why there are practically no other Mac OS X or iOS developer tools on the market today. There just isn’t any room for third parties for it to make economic sense to develop expensive developer tools.

The only other serious IDE for Mac OS X and iOS development is JetBrain’s AppCode and I’d recommend that every serious Apple developer should own a copy. While Xcode has evolved into a powerful and mostly stable tool, Apple has a lot of blindspots and Xcode is in many areas (at least) 15 years behind the top of crop. AppCode isn’t.

JetBrains is the powerhouse of Java development tools and they represent everything that Apple does not. Where Apple is closed, secretive and has a very paternalistic approach to its developer community, JetBrains is open, transparent, friendly and as cross-platform as it is possible to be.

The advantages for an Apple developer such as myself is that you get a peak at the world beyond Apple’s strictly enforced white room monoculture. Using AppCode is as much about growing as a developer as it is about efficiently developing software.

JetBrains offers IDEs that support nearly every language that is available and the more outrageously new and niche a language is, the more likely that JetBrains has a tool for it. This means that once you get used to the basic IDE concepts, you can take that expertise and use it for developing in other languages, on other platforms (Android, Windows, Web) and with other technology stacks.

I use WebStorm for my own website development, RubyMine for web app stuff and IntelliJ IDEA for learning functional programming in Scala. If I ever wanted to learn CoffeeScript, Dart or Haskell I know I’d be covered there too. On top of this JetBrains’ plug-in technology makes adding support for the latest and greatest open source technologies a breeze and JetBrains are very good at keeping an eye open for exiting new technologies. There’s a good chance that the first you hear about a new technology is by looking at JetBrains’ product release notes.

The AppCode IDE itself is very much in the mold of other Java development environments. The IDE can do everything and more, but it is also very busy and a long way from the pared-down minimalistic Apple aesthetic. It’s a nerdy power tool more than a philosophical statement.

JetBrains is rightly famous for their language parsing and refactoring acumen, so their IDEs are chock full of “intelligent” features. Not the kind of “intelligent” that makes everything harder, but the actual intelligent kind.

Navigating in AppCode is much more powerful than in Xcode. The gutter contains a myriad of options that will take you from method implementation to declarations and vice versa. You can also click and hold from class definitions to jump to super- and sub-classes, get in-line help and auto-fixing for commons problems. The as-you-type code analyzer finds potential problems and standard fixes, the code reformatting options are powerful and easily accessible. The intelligence extends to seamlessly into finding all places a piece of code is actually used rather than having to rely on text searches.

Best of all, however, AppCode can make changes to associated files without leaving the current file. The annoying roundtrip between implementation and header files that keeps interrupting your train of thought in Xcode can be wholly avoided. You write the implementation for a method and AppCode just offers you the ability to declare said method in the header with a single click without ever taking your eyes of the code you are busy writing.

Working in AppCode you constantly find yourself wondering why Apple can’t just do this. If it seems obvious, it’s in AppCode. Unfortunately this is rarely true for Xcode.

Refactoring is part and parcel of the AppCode experience and backed so far into the IDE that it becomes a nearly invisible part of your development. If you are used to refactoring in Xcode, you are likely to be non-plussed by AppCode’s refactoring support. Where Xcode makes a huge deal out of every refactoring: taking a snapshot, making you validate a thousand changes and more likely than not failing bang in the middle of the refactoring, AppCode just makes the changes with no fuss whatsoever. The first time I used the renaming refactoring in AppCode, I was wondering what I was doing wrong. I typed the new name into the red highlighted area and nothing happened! How do you terminate the editing? In fact, AppCode had already done the project-wide refactoring. Why make a fuss about it? Why could it fail? Why beach-ball for a few seconds? Why indeed?

AppCode enables you to work in a completely different manner to Xcode. Say you are into Test-Driven Development. Write the test cases first. When you instantiate your target class in the test class, AppCode will tell you that the class does not yet exist. A single click solves the problem by creating the class for you. As you write your tests, you can one-click to add method declarations and empty implementations. When you’ve finished with your test cases, there’ll be .m and .h files with complete stub implementations all without you ever leaving the test case implementation file.

Another big differences with Xcode is that where Apple knows everything best and either offers no customization or forces you to comply with their guidelines, JetBrains puts you in charge. Almost every aspect of the IDE is fully customizable: you can define your own coding style, which will cause AppCode to use your specific style to create stubs. You can even decide to reformat your code automatically before checking it into source control. You can (obviously) choose your own source code management system, add CocoaPods support, edit and preview HTML, CSS, Compass, TypeScript, JavaScript, files or add your own selection of plug-ins. In short, JetBrains is for grown-ups that like taking their own decisions.

Similarly, if you’ve ever felt the frustration of never being able to talk to anybody at Apple about Xcode, you will find the JetBrains support team a breath of fresh air. Something not working? Something not supported? Something you’d like to see added? Just drop them a line and an actual person will reply to you; better yet that person will be an approachable, open-minded fellow developer intent on helping you out. With JetBrains you’re the customer and you know best.

Seriously, just give it a shot. If only for a breath of fresh air.

The unbearable fragility of modern Mac OS X development

There I’ve done it again: I shipped a broken A Better Finder Rename release despite doubling down on build system verification, code signing requirements validation and gatekeeper acceptance checks, automation, quality assurance measures, etc.

Only in October, I had a similar issue. Luckily that time around it only took a few minutes to become aware of the problem and a few hours to ship a fix so very few users were affected. Right now I don’t know how many users were affected by the “botched” A Better Finder Rename 10.01 release.

This didn’t use to happen. Despite the fact that I did not spend nearly as much time ensuring that everything worked properly with the release management. Nor am I alone in this situation. Lots of big as well as small developers have recently shipped similarly compromised releases.

The situation on the Mac App Store is much, much worse. Nobody other than Apple knows how many Mac App Store customers were affected by the recent MAS certificate fiasco that had the distinction of making it all the way into the pages of Fortune magazine.

The truth is that Mac OS X development has become so very fragile.

The reasons for this are manifold and diverse but boil down to: too much change, too little communication, too much complexity and finally too little change management and quality control at Apple.

The recent Mac App Store (MAS) fiasco that left many (1% of Mac App Store users? 100%? Nobody knows) users unable to use their apps purchased from the Mac App Store was down to Apple’s root certificate expiring. This was a planned event: certificates are used for digitally signing applications and they are only valid for a particular period of time, after which they need to be replaced with new certificates.

When the Mac App Store certificate expired, it was replaced with a new certificate but there were two problems. First, the now expired certificate was still cached by some of Apple’s servers: when Mac OS X opens an application it checks its signature, which in the end is guaranteed by Apple’s root certificate. Since this was no longer valid, Mac OS X refused to launch them and reported them as “broken”, leaving users and developers equally baffled. After far too long, Apple investigated the problem and emptied their caches which made the problem go away.

The second problem which was not solved by updating the caches, was due to Apple also replacing the certificate with a new, higher security version; of course without telling anybody. The new certificate could not be verified with the old version of OpenSSL that was used in the receipt checking code of many shipping apps.

When Apple created the Mac App Store, it provided a “receipt” that each application should check to see whether it has been properly bought on the Mac App Store. This is just a signed file that contains details about what was bought and when. Instead of doing the obvious thing, which would have been to provide developers with an API for checking the validity of the receipt against Apple’s own rules, they just publishing snippets of sample code so that each developer could “roll their own” verification code. Supposedly this was for added security (by not providing a single point of failure), but it seems more likely that they couldn’t be bothered to ship an API just for the Mac App Store.

This decision came back to haunt them, because most developers are not crypto experts and so had to rely on developer contributed code to check their app’s receipts. Once this worked properly, most developers wouldn’t dream of touching the code again.. which is how it came to pass that many, quite possibly a majority, of Mac App Store apps shipped with the same old receipt checking code in 2015 that they originally shipped with in 2010(?). This was fixed by Apple revoking the new style certificate and downgrading it to the old standard.

For once, I had been ahead of the curve and had recently updated all the receipt code in my applications (no small feat) and I have yet to hear from any customers who had problems.

Just before the Mac App Store fiasco, however, many non-MAS had also shipped with broken auto-update functionality.

Apple does not offer any auto-update facility for applications that are not on the Mac App Store, which lead to Andy Matuschak’s “Sparkle” framework becoming the de-facto standard for adding an auto-update features to Mac applications.

Driven by abuse of some HTTP communications in iOS apps, Apple decided that in iOS 9 it would by default opt all developers into using only (more secure) HTTPS connections within their applications. What is good for iOS 9 can’t be bad for Mac OS X 10.11 El Capitan, so Mac applications also got opted into this scheme.

Unfortunately, that broke Sparkle for applications which do not point to HTTPS “app casts” such as mine. I have long resisted installing my own HTTPS certificates because I was worried about messing up the expiry periods, etc.. apparently just the way that Apple did with the Mac App Store certificates.

Most developers will have been unaware of the change since Apple never announced it, but I had happened to see the WWDC conference videos that mentioned this in passing. Unfortunately, nothing “clicked” in my head when I heard this. My applications do not communicate with a server anywhere and I thus thought that this was not something I had to worry about. I had forgotten that Sparkle might use this internally.

Fortunately, I caught this at 6AM when I released A Better Finder Rename 10 final. I was just doing a normally completely redundant check through all the features of the program when I noticed that the new version failed when trying to check for updates. By 8AM, I had identified and fixed the problem so that very few people indeed could have been caught out by it. That was luck though.

The nefarious element here was that applications were opted in automatically and silently. Before 10.11 El Capitan was installed on your Mac, my applications updated just fine. Afterwards, they no longer did. Just because they were on El Capitan. Gee thanks!

Of course, this would not have happened if I hadn’t built A Better Finder Rename 10 with the Mac OS X 10.11 SDK (Software Development Kit) at the last moment.

It is somewhat crazy for a developer to change the SDK that s/he builds a brand-new version of their software against in the middle of the beta phase. Changing the SDK always introduces errors because the entire environment in which the code executes is changed. This may bring out bugs that were already present; things that should never have worked, but worked just because the API happened not to trigger the bug. It also introduces bugs that are just part of the new SDK and that you now have to work around. Changing SDKs makes existing programs fragile.

I’m very conservative when it comes to changing SDKs because I’m well aware of the risks. That’s why I’ve been building my code against older SDKs for the past 15 years. A Better Finder Rename 10 was built against the Mac OS X 10.7 SDK which is forwards-compatible with newer versions of Mac OS X.

The main reason for doing so, is that I wanted to be certain that I didn’t accidentally break A Better Finder Rename on older systems, which brings us to the next problem with Mac OS X development.

Xcode lets you specify a “deployment target”, for instance 10.7, while building with a newer SDK. This is the recommended way of developing on Mac OS X and keeping backwards compatibility. Xcode will, however, happily let you use APIs that are not compatible with your deployment target and thereby ensure that your application will crash on anything other than the latest Mac OS X.

In fact, Xcode encourages you to use the latest features that are not backwards compatible and will rewrite your code for you if you let it, so that it will crash. It will give you “deprecation warnings” for any API usage that is not in the latest SDK and resolving those warnings is likely to break backwards compatibly as well. Of course, you won’t know this until you run it on the old Mac OS X version.

Now which developer can afford to keep testing rigs with 10.7, 10.8, 10.9 and 10.10? Never mind spend the time re-testing every change on multiple platforms for each change?

Thus I happily built with the 10.7 SDK. Apple did not make this easy by not shipping the old SDKs with Xcode, but you could manually install them and they would work just fine.

Imagine my surprise after installing Xcode 7 and finding out that this no longer worked. The only workable solution was to build against the 10.11 SDK, so jumping forwards not one but 4 SDK versions. A bunch of code wouldn’t compile any longer because the libraries were gone. Luckily the receipt checking code was amongst those, so it got modernised just in time to avoid the Mac App Store receipt fiasco.

Nonetheless, now my entire code base had become fragile and largely un-tested between the last beta release and the final shipping product. Nightmare!

On top of that was it still even 10.7 compatible? or indeed 10.10 compatible? Just quickly running it on older systems wouldn’t provide more than a little additional confidence since it’s impossible to go through every code path of a complex product.

After installing virtual machines to test on, I still couldn’t be 100% certain. The solution came in the form of deploymate, a now truly essential developer tool which does what Xcode can’t do: check that API usage is compatible with the deployment target.

I have since spent many weeks trying to ensure that I won’t run into the same problems again by adding (additional) automated verification processes to my build system. My build system now runs the built product through SDK compatibility checking courtesy of deploymate, code signing validation and gatekeeper verifications on each build. I’m still working though deprecation warnings and the like and my code base will soon be bullet proofed at least until the next forced changes arrive.

You’d think that this was a long enough list of problems for one year, but this still does not account for Apple also changing the code signing rules (once again) earlier in the year (in a point update of 10.10 no less). This time it affected how resources and frameworks are signed. So applications that were signed correctly for years, now suddenly became incorrectly signed and Mac OS X would refuse to launch them because they were “broken”.

All this points to the underlying issues with the current spade of fragility of Mac applications: Apple keeps changing the status quo and neither it, nor developers have any chance of keeping up.

Apple’s own applications are full of bugs now. None more so than Xcode, which is both the lynch pin of all Mac OS X, iOS, watchOS and tvOS development and no doubt Apple most fragile app offering. Xcode is in beta at least 6 months a year and never really stabilises in between. Each new version has new “improvements” to code signing, app store uploading, verification code, etc.. and each new version breaks existing code and introduces its very own new bugs and crashes. From one day to the next, you don’t know as a developer whether your code works or not. Existing code that had worked fine on Friday evening, no longer works on Monday morning. Worse, chances are that you are not hunting for your own bugs, but those in your development tools, the operating system or Apple supplied SDKs.

All this is driven by the one-release-a-year schedule that Apple has imposed on itself. This leaves all of Apple’s software in various stages of brokenness. When Apple’s own staff cannot deal with this constantly shifting environment, how are third party developers supposed to?

Case in point: Apple’s own apps are not all iOS 9 compatible yet. Many don’t support the iPad Pro’s new native resolution yet. Some have gained Apple Watch extensions, but most haven’t.

Reliability is a property of a system that is changed slowly and deliberately and where all constitute parts are themselves reliable. The Mac and all other Apple platforms are currently undergoing the worst dip in reliability since Mac OS X was introduced.

Apple is pushing out half-baked annual releases of all its software products, as well as apparently completely unmanaged changes to policies, external rules and cloud services at an ever more frenetic pace.

These could be written off as temporary “growing pains”, but the big question is: Do all these annual updates equate to real progress?

When I switch on my Mac today, I use it for much the same things that I used it for 10 years ago. A lot has changed. Cumulatively Mac OS X 10.11 El Capitan is somewhat better than 10.6 Snow Leopard.. and yet if you discount cosmetic changes and new hardware, nothing much has changed. Certainly nothing much has actually improved.

I can’t help thinking that if we had had 2 or possibly 3 Mac OS X updates instead of 5 over those last 5 years, we’d be in a much better shape now. Apple and developers would have time to provide user benefits and rock solid reliability rather than just endlessly chasing their own tail.

The beauty of the Mac used to be that it just worked. I want to get back to that.

Some thoughts about Optionals in Swift

Like many Mac and iOS developers, I haven’t yet made the jump from Objective-C to Swift, nor will I make it in the near term.

Just like most other developers, I am slowly putting a little bit of Swift code into new projects to get a feel for the language and I’m experimenting with its new features to be ready when it becomes a more practical proposition.

At the moment, I’m trying to come to grips with “optionals“, a construct that I have never seen in any language I’ve used before.

Essentially, “optionals” are shoeboxes that may or may not contain a value. You can find out whether they contain a value and if so, you can get the value out by “unwrapping it” and if it contains a mutable type, you can change it.

This replaces the C-style pointers that may either point to the data belonging to an object (in the largest sense of the word) or have a “nil” or “NULL” value that signifies “no object”.

Optionals are a clear improvement on null pointers or nil types in at least two significant ways:

  1. Optionals eliminate a whole class of common bugs by forcing developers to think about “what happens if this method does not return a valid object/ value”
  2. Optionals help make Swift code faster by making it easier and safer to optimize code; optimizers do not have to worry about edge cases where the value may be empty.

The price for those two advantages seems very steep, however.

First, not dealing with potentially invalid objects being returned prevents the code from being compiled in the first place. You cannot not deal with the problem.

Secondly, optionals require developers to learn a host of new concepts. This may be a good thing in the long term, but it will slow adoption as you can’t just dive into the language without understanding optionals and you have little hope of ever having come across them before (unless you are beta testing Rust that is).

Using optionals is also really, really cumbersome. Unwrapping each and every optional makes even trivial programs balloon to many times their normal size. This is why Apple has thoughtfully added a lot of syntactic sugar to make the medicine go down that much more easily.

This syntactic sugar (chaining, etc.) makes your code smaller and cleaner and you can almost forget that it is there at all when you read it. Unfortunately that isn’t the case when you write the code and it does also add a whole layer of complexity making Swift harder to learn and understand.

Optionals also do not map onto anything in Objective-C and as there are no native libraries yet, “implicitly unwrapped optionals” abound and feel very much like a backwards-compatibility hack and a solution to a problem you wouldn’t have without optionals.

So there is a price to be paid for beauty, performance and safety.

That needn’t be a show stopper. You get nothing for free in life and the result may be worth the effort.

Unfortunately, when I’m actually using Swift, I find that I don’t seem to be getting much value out of optionals.

Firstly, there’s the fact that now instead of trying to express what I want to do, I find myself wondering about whether I need to unwrap my optionals? and what might be the best syntax for expressing the unwrapping? where should it be done? now or later? Why does this code not compile?

Admittedly with a little bit of practice, this will probably largely go away. There will always be a  cognitive overhead associated with it, but it will that overhead is likely to get smaller with time. I certainly won’t miss having to check the location member of the return value of rangeOfString: for NSNotFound.

The real killer for me is, however, that I now find all those places in my code where I need to think about what to do when the optional is empty .. and I find that there’s not much to be done.

What can you do when you are loading an image from your app bundle and it is not there? There only seem to be two options:

  1. raise an exception, crash or display an alert that the user can’t do anything about
  2. do nothing

Crashing is great, because now there’s a crash report being generated and I know that something has gone wrong in the production version and I can work out what the cause is and correct it. Crashing early is best practice. It certainly beats crashing later.

Doing nothing might be okay in many situations, because it might not be so bad. It is in fact what Objective-C does most of the time in this situation. It just ignores the problem, the application keeps running (but not necessarily working) or it simply crashes somewhere down the line.

Herein lies the problem with optionals. They force you to deal with potential problems that you can’t do anything intelligent about. Worse, because of the absence of an exception mechanism you need to deal with exceptions in situ, polluting your beautiful code with non-sensical error handling code.

So, what I’m wondering is: “Is there any value in explicitly dealing with problems that you can’t do anything about?”