iOS UI Testing Nick McConnell nicholasgmcconnell@gmail.com - - PowerPoint PPT Presentation

iOS UI Testing

Nick McConnell

nicholasgmcconnell@gmail.com https://www.linkedin.com/in/nickgmcconnell

Intro to UI Testing

• Automated UI Testing, that is!
• Xcode 7, iOS9 only
• Similar UIAutomation (which used javascript) >

this has been mostly dropped

• Uses accessibility
• “Blackbox” approach (?)

You are going to like:

• Runs externally, so you are running the exact same code
• Some thoughtful options - querying of UI elements
• Recording :)
• Can use swift or Objective-C
• Can debug
• Can help with repeated “manual” testing
• Managed in the same way as XCTest Unit Tests
• Relatively fast (caveats, can remove animations)
• Improvements through the beta 7’s

Feels natural (kind of)

UI Testing

(not a silver bullet) (a silver bullet)

One of Many Ways of Testing…

• QA Testing
• Black box testing
• White box testing
• Gray box testing
• Manual Testing
• Automated Testing
• Regression Testing
• Acceptance Testing
• Performance Testing
• Scalability Testing
• Unit Testing
• UI Testing
• Stability Testing
• Smoke Testing
• Sanity Testing
• Integration Testing
• System Testing
• Compatibility Testing
• Alpha Testing
• Beta Testing
• Monkey Testing
• Gorilla Testing
• Baboon Testing
• Security Testing
• Destructive Testing
• A/B Testing
• Functional Testing
• Component Testing
• Exploratory Testing
• Load Testing
• Negative Testing
• Internationalization Testing
• Non-functional Testing
• Stress Testing
• Usability Testing
• Ad-hoc Testing

A Common Perspective….

Compare

• UI Automation
• Javascript
• Uses Accessibility
• No longer supported
• KIF
• objective-C
• developed by Square
• uses undocumented API (future issues?)
• “Breaks with every major iOS revision”, “slow”
• Possibly more flexible, but complexities to consider
• Calabash
• Uses Cucumber
• Android
• Appium
• Supports several languages
• Android

Management

• Separate Target (can add automatically when creating new project or later)
• Runs Externally from App
• Sim or Physical
• Source control easy (code part of project)
• Right-click on the method name…. run and disable/enable
• Test Navigator (right-click… run test)
• Report Navigator (right-click… jump to reports)
• Report Navigator - Check screenshots
• Schemes to make test execution.
• Keyboard shortcuts: ⌘U to run all, ⌃⌥⌘U to run individual tests

Basics

• 2 classes: XCUIElementQuery & XCUIElement
• Queries are stored paths to elements
• Elements are the singular paths to UI Objects obtained

through queries.

• Elements are NOT UIViews, they are “proxies”
• Queries and Elements are only resolved when needed
• Elements have types that map to UI Objects but are not

“class typed” (Buttons, Images, StaticTexts, Cells, Tables)

Basic Queries

USES THE VEIW HIERARCHY >>>

descendantsMatchingType (type) > All the way down the tree childrenMatchingType (type) > One level down only containingType(type, identifier) > Looks below Oh yeah! Convenience methods instead of descendantsMatchingType… .buttons .tables .staticTexts (Labels) .cells .alerts .otherElements (Views - useful) .images .navigationBars

Table App NavBar Cell Cell Label Label Button

From Query to Element

by Text .staticTexts[“Ape”] .containingType(.StaticText, “Monkey”) >>> Will use text labels, accessibility IDs, “processed”image names by Index .elementBoundByIndex(index) Using Predicate .containingPredicate(NSPredicate(format: "label contains[c] 'hello'")) Assume One (will fail if more) .element

Actions on Elements

Is it there? .exists() Taps etc .tap(), doubleTap(), twoFingerTap(), .pressForDuration(duration) etc Swipes .swipeUp(), .swipeDown(), .swipeLeft() etc Other .rotate(rotation,velocity), .pinchWithScale(scale,velocity) Count a query .count()

First Query

let app = XCUIApplication() let labels = app.descendantsMatchingType(.StaticText) let label = labels[“Baboon”] XCTAssertTrue (label.exists())

OR

XCTAssertTrue (app.StaticTexts[“Baboon”].exists())

Record

• Magic Red Button
• Can you use to add to existing test code
• Caution: Sometimes creates code that doesn’t run

(eg system alerts)

• Caution: Sometimes creates brittle, awkward
• code. Consider rewriting.
• Great for getting started with UI Testing

Recording Craziness

app.childrenMatchingType(.Window).elemen tBoundByIndex(0).childrenMatchingType(.O ther).element.childrenMatchingType(.Othe r).element.childrenMatchingType(.Other). element.childrenMatchingType(.Other).ele ment.childrenMatchingType(.Other).elemen t.childrenMatchingType(.Other).elementBo undByIndex(2).tables.childrenMatchingTyp e(.Cell).elementBoundByIndex(4).children MatchingType(.Button).elementBoundByInde x(0).tap()

Surprisingly…

• No official docs for XCUIElement etc
• Can’t get access to state, color, font
• Can’t check directly for views, activity indicator etc (caveats)
• Can’t call any application classes directly
• UI Automation is not for non-techies
• No code coverage (maybe not surprising)
• Some instability
• What you record doesn’t always run
• v1 > Hope there is more to come

Real World

• Logic paths based on signed-on or not etc
• Look at backend and database’s data stability
• Start from a solid refresh point: can delete app from sim:
• xcrun simctl uninstall booted com.fueled.OnyDo || true
• If adding data, look at repeatability. May need a clean-

up step afterwards. May need separate DB/ environment.

• Could look to Mocks (change your code)

Command Line & CI

• Command Line: xcodebuild test
• destination to pick device.
• Xbots - good integration with unit, UI, code coverage and screenshots
• Jenkins (use agent not daemon, with OCUnit2Junit XSLT can trend,

can give access to screenshots)

• Bitrise - just works (basic)
• Bamboo - ???
• UI Testing is generally less stable than Unit testing
• Screen shots

Long Term Value & Maintenance

• Would guess 90% of historically written automated UI Tests don’t survive

further than the first launch.

• Automated UI Test code is just code of course, and code needs TLC by

coders.

• Consider how easy your UI Test code will respond to future UI changes.
• Consider directly rewriting “recorded” code. Good start but often very

brittle.

• Break up tests into separate methods for each step… encapsulate sign-in

process

• Consider design patterns such as Page Object pattern (http://

code.tutsplus.com/articles/maintainable-automated-ui-tests--net-35089

• Whole team / org decision

Final

For all the caveats, UI Testing may be the Automated UI Test tool you actually want to use. It’s easy to get into and it’s right there in Xcode. And with that, you may end up with real, valuable UI Tests that will become part of your daily workflow.

Misc Comments

• WWDC 2015: https://developer.apple.com/videos/wwdc/2015/?id=406
• Pinching and Zooming, Hardware buttons, UIPicker…. new in the final

Beta.

• Performance Tests - possible
• Test case order is random
• Can use it as a quick-start if you are repeating a lot of manual tests

(command line)

• System Alerts - automaticallu clicks OK/Allow
• Can even “join in” even running UI Tests