Testing in iOS is something that we as developers don’t really think about, or do. Writing tests can help us write code that lasts for years to come. But, this post isn’t about why you should write tests, it’s about how to use GHUnit and KIF with Jenkins CI.
There are two main types of testing frameworks, Unit and UI. OCUnit, which ships with Xcode, and GHUnit are both Unit testing frameworks. UIAutomation, Frank, and KIF are examples are UI testing frameworks.
Unit tests are used to test vertical, isolated slices of functionality. Your app won’t actually be running when these tests are run. Instead, you’ll only import and create objects you want to test directly.
GHUnit has some fantastic setup steps. The most important thing to note is that GHUnit has its own target, which is not a duplicate target of your app. This means that your app will not actually be running when the tests run.
All of your tests will be subclasses of
GHTestCase. And you can group these together by creating a
GHTestGroup. If you’ve used
OCUnit before, this should all sound familar.
Each test case will determine sucess or failure by using assert macros (you can see a full list of them on this page).
GHUnit also happens to have a great guide for setting up tests using CI. Note that if you’re running Xcode 4.5 or newer there is a bug where the old way of running tests does not work. This is being actively tracked in GitHub Issues for the project, as well as on openradar.
GHUnit also will output JUnit test results, which Jenkins can track over time. This is great for keeping track of code stability, and for finding trends in how well your tests are doing.
Arguably the hardest part of automated testing is the fact that it’s been broken since Developer Preview 3 of Xcode 4.5. This means that you have to work around the command line tools in order to get your tests to run. It’s something that is actively being tracked, and there are solutions, but they are far from ideal.
Since your app won’t be running during these tests, everything you test is very isolated. That might be great for your app, but not all apps can be tested thoroughly with only small pieces running at a time.
Also, since your app is not actually running, running UI tests with GHUnit is not ideal.
KIF is a framework created by the folks at Square. It uses the
<UIAccessibility> protocol to interact with your app, which is a private API. When adding KIF to your project, you have the choice of using git submodules, or Cocoapods.
KIF setup is fairly simple, and is documented in the project’s README. The main difference between KIF and GHUnit is that KIF duplicates your application target, so all of your source code is added to KIF.
The Test Controller determines at runtime what tests to run, if you don’t want to run all of them (which you won’t). You can choose to run different sets of tests based on just about anything, the iOS version of the device, the device idiom, etc.
KIFTestScenario & KIFTestStep
The bulk of your test writing will be creating scenarios, like “user logs in” or “user visits cart”, which are comprised of steps. To create reuseable steps, you subclass
KIFTestStep, and then can define your step using blocks. Likewise, you can subclass
In the KIF README there is a section on Automation that provides instructions, and a bash script, for getting it running. However, I’ve found that
ios-sim works a bit better, but both work.
KIF doesn’t automatically output test results in any useable format. There’s currently a pull request open to get that in. Or, you could get a patch from an older pull request, which is the one I’ve been using for months. Either way, you need to fork KIF.
Here’s a simple command to run KIF with an .app file stored in
$APPFILE and then save the JUnit xml:
/usr/local/bin/ios-sim launch $APPFILE --family ipad > /tmp/KIF-ipad-$$.out 2>&1
cp "`grep "JUNIT XML RESULTS AT " /tmp/KIF-ipad-$$.out | sed 's/.*JUNIT XML RESULTS AT //'`" 'test-reports/KIF-ipad-results.xml'
KIF is fast
KIF runs at processer speed. It doesn’t have a “reaction time” like people do, so KIF will try to find that view on the screen before your navigation push animation finishes. To compensate for this, you’ll end up doing a lot of:
[scenario addStep:[KIFTestStep stepToWaitForTimeInterval:1 description:@"wait"]];
[scenario addStep:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:@"Table"]];
Now, this isn’t necessarily a bad thing. But it will take time to get in the habit of thinking to add waits into your tests. And tests will take longer to run, of course.
KIF can’t recover
KIF has a handy way of adding a step, or set of steps, to be run before every scenario:
[KIFTestScenario setDefaultStepsToSetUp:[KIFTestStep setupSteps]];
However, in these default setup steps you need to be able to recover from a failure at any point in your app. If you don’t recover, then there’ll be a domino-effect of failed tests. This will mean having methods available to esentially reset the application state.
You’re code coverage will not be evenly distributed
If your app requires a login for example, it’s likely you’ll have something like this in many scenarios:
[scenario addStepsFromArray:[LoginTestStep stepsToLoginWithEmail:TEST_EMAIL password:TEST_PASSWORD]];
Because you want to “reset the application state” between scenarios, you’ll end up testing the parts of your code that deal with common actions more frequently than other parts of your app. This is both a pro and a con; the parts you test over and over will definitely be rock solid, but that means there will be parts of your ap that won’t be tested nearly as much.
KIF can’t see off the screen
Since KIF relies on
<UIAccessibility> and that relies on view drawing, KIF can’t see things that are off the screen. You’ll end up frequently doing something like this:
[scenario addStep:[KIFTestStep stepToScrollToItemWithAccessibilityLabel:@"Settings"]];
KIF is a great framework, but Square has an “Individual Contributor License Agreement (CLA)” that you must sign before any of your code can go into core. This means that there are a number of great things that developers have made, but never got merged in (like the JUnit code).
Does writing tests help you write better code? Yes.
Will testing add to development time? Yes.
Is it worth the extra time? Yes.
If you’re interested in seeing some sample tests, head over to my Github repo.
As a mobile developer, I use a lot of APIs. I’ve worked with some really great ones, and some ones with lots of room for improvement. Here are a few lessons I’ve learned that hopefully you can think about the next time you are developing, or working with, a new API.
For the sake of this argument, an API is any way a 3rd party can communicate with your software. This includes both web services, and releasing a library that other developers can add to their own code base.
Lesson 3: Follow Conventions
Conventions have been established for many reasons, most often because they just make sense. If you’re going down a path that you haven’t been before, following conventions is a good place to start. As a developer, you’ll be able to learn from the wisdom of many before you, and you’ll end up producing a product of greater quality than if you were to code in a box.
Another important reason to follow conventions is the fact that problems will be easier to debug. The last thing you want, when letting others interact with your software, is for you to do hours of support (which take away from development time). This will greatly help developers working with your API.
If you are going to establish your own convention, which you will for a variety of reasons, make sure you are consistent. Adding paging, for instance, should be the same across all of your calls that support it; don’t go changing one API in a set to be different. Consistency is a big part of following conventions, regardless of who defined them.
Lesson 2: Don’t Be Clever
As programmers, we like to think we’re pretty smart. We like to find the best solution to a problem. Sometimes, however, this leads to us being clever. Clever is a pitfall, and it makes everyone’s life more complicated.
When making an API it’s your responsibility to make things as dead simple as possible. Consumers of your API want access to your data as easily and quickly as possible. Keeping things simple will make everyone happy, especially in the long run.
Lesson 1: Document, Document, Document
Did I say document? The purpose of an API is to be reused. If it’s not documented well, then it will be very difficult to use in the future. Expect the code you’re writing now to have at least a 5 year lifespan. Will you still be working on this one thing in 5 years? Probably not; you may not even be at the same company by then.
Writing good documentation is a requirement these days when releasing an API to the public. Documenting code is a skill that every developer should have, so take this opportunity to better your skills at it.
Screenshots and examples are also good things to include. If you want to go above and beyond, include an API console on your website, so people can play with it without the mess of a local client.
Lesson 0: Expect the Unexpected
Once your code goes live, people will start using it. And since we’ve established a timeline of at least 5 years, whatever you build now will be used in a different purpose in the future.
If you follow the first 3 lessons, this one will be easy to handle. Documentation is by far the most important thing to have, so make sure it’s easy to find and well written.
In summary, APIs are becoming a vital part of your software stack. These are just a few lessons I’ve learned over the years I’ve been working with APIs. Obviously, some rules are meant to be broken. We all work in different ways. These are just some tips to help you in your creation!
I’m really excited about this new layout. It’s much more modern than the old one, but keep in mind this is still a work in progress. There are a few things I have left to fix, especially on mobile *cough*Android*cough*.
One thing to note. I have removed my ‘Work’ section entirely. Why?
There are great tools out there, now, that display my work experience in a more cohesive manner, as well as show things like connections and endorsements. It’s also one less dependency for me to keep updated. Honestly, who needs another thing to update these days? Checkout my Linkedin or Zerply profiles if that’s really what you’re looking for.
Oh yeah, and the ‘About’ page is gone too. Yep, 3 pages down to one. And it feels so good.
If you have any feedback, or find an issue with the new layout, please email me!