Blog

Posted August 8, 2014

Appium Bootcamp – Chapter 6: Run Your Tests

[UPDATE- November 2019] You can find the newest Get Started with Appium white papers here: Get Started with Appium- Java and Get Started with Appium- Ruby

This is the sixth post in a series called Appium Bootcamp by noted Selenium expert Dave Haeffner.

Dave recently immersed himself in the open source Appium project and collaborated with leading Appium contributor Matthew Edwards to bring us this material. Appium Bootcamp is for those who are brand new to mobile test automation with Appium. No familiarity with Selenium is required, although it may be useful. This is the sixth of eight posts; two new posts will be released each week.

Now that we have our tests written, refactored, and running locally it's time to make them simple to launch by wrapping them with a command-line executor. After that, we'll be able to easily add in the ability to run them in the cloud.

Quick Setup

appium_lib comes pre-wired with the ability to run our tests in Sauce Labs, but we're still going to need two additional libraries to accomplish everything; rake for command-line execution, and sauce_whisk for some additional tasks not covered by appium_lib.

Let's add these to our Gemfile and run bundle install.

1
# filename: Gemfile
2

3
source 'https://rubygems.org'
4

5
gem 'rspec', '~> 3.0.0'
6
gem 'appium_lib', '~> 4.0.0'
7
gem 'appium_console', '~> 1.0.1'
8
gem 'rake', '~> 10.3.2'
9
gem 'sauce_whisk', '~> 0.0.13'

Simple Rake Tasks

Now that we have our requisite libraries let's create a new file in the project root called Rakefile and add tasks to launch our tests.

1
# filename: Rakefile
2

3
desc 'Run iOS tests'
4
task :ios do
5
  Dir.chdir 'ios'
6
  exec 'rspec'
7
end
8

9
desc 'Run Android tests'
10
task :android do
11
  Dir.chdir 'android'
12
  exec 'rspec'
13
end

Notice that the syntax in this file reads a lot like Ruby -- that's because it is (along with some Rake specific syntax). For a primer on Rake, read this.

In this file we've created two tasks. One to run our iOS tests, and another for the Android tests. Each task changes directories into the correct device folder (e.g., Dir.chdir) and then launches the tests (e.g., exec 'rspec').

If we save this file and run rake -T from the command-line, we will see these tasks listed along with their descriptions.

> rake -T rake android # Run Android tests rake ios # Run iOS tests

If we run either of these tasks (e.g., rake android or rake ios), they will execute the tests locally for each of the devices.

Running Your Tests In Sauce

As I mentioned before, appium_lib comes with the ability to run Appium tests in Sauce Labs. We just need to specify a Sauce account username and access key. To obtain an access key, you first need to have an account (if you don't have one you can create a free trial one here). After that, log into the account and go to the bottom left of your dashboard; your access key will be listed there.

We'll also need to make our apps available to Sauce. This can be accomplished by either uploading the app to Sauce, or, making the app available from a publicly available URL. The prior approach is easy enough to accomplish with the help of sauce_whisk.

Let's go ahead and update our spec_helper.rb to add in this new upload capability (along with a couple of other bits).

1
# filename: common/spec_helper.rb
2

3
require 'rspec'
4
require 'appium_lib'
5
require 'sauce_whisk'
6

7
def using_sauce
8
  user = ENV['SAUCE_USERNAME']
9
  key  = ENV['SAUCE_ACCESS_KEY']
10
  user && !user.empty? && key && !key.empty?
11
end
12

13
def upload_app
14
  storage = SauceWhisk::Storage.new
15
  app = @caps[:caps][:app]
16
  storage.upload app
17

18
  @caps[:caps][:app] = "sauce-storage:#{File.basename(app)}"
19
end
20

21
def setup_driver
22
  return if $driver
23
  @caps = Appium.load_appium_txt file: File.join(Dir.pwd, 'appium.txt')
24
  if using_sauce
25
    upload_app
26
    @caps[:caps].delete :avd # re: https://github.com/appium/ruby\_lib/issues/241
27
  end
28
  Appium::Driver.new @caps
29
end
30

31
def promote_methods
32
  Appium.promote_singleton_appium_methods Pages
33
  Appium.promote_appium_methods RSpec::Core::ExampleGroup
34
end
35

36
setup_driver
37
promote_methods
38

39
RSpec.configure do |config|
40

41
  config.before(:each) do
42
    $driver.start_driver
43
  end
44

45
  config.after(:each) do
46
    driver_quit
47
  end
48

49
end

Near the top of the file we pull in sauce_whisk. We then add in a couple of helper methods (using_sauce and upload_app). using_sauce checks to see if Sauce credentials have been set properly. upload_app uploads the application from local disk and then updates the capabilities to reference the path to the app on Sauce's storage.

We put these to use in setup_driver by wrapping them in a conditional to see if we are using Sauce. If so, we upload the app. We're also removing the avd capability since it will cause issues with our Sauce run if we keep it in.

Next we'll need to update our appium.txt files so they'll play nice with Sauce.

1
# filename: android/appium.txt
2

3
[caps]
4
appium-version = "1.2.0"
5
deviceName = "Android"
6
platformName = "Android"
7
platformVersion = "4.3"
8
app = "../../../apps/api.apk"
9
avd = "training"
10

11
[appium_lib]
12
require = ["./spec/requires.rb"]
13
# filename: ios/appium.txt
14

15
[caps]
16
appium-version = "1.2.0"
17
deviceName = "iPhone Simulator"
18
platformName = "ios"
19
platformVersion = "7.1"
20
app = "../../../apps/UICatalog.app.zip"
21

22
[appium_lib]
23
require = ["./spec/requires.rb"]

In order to work with Sauce we need to specify the appium-version and the platformVersion. Everything else stays the same. You can see a full list of Sauce's supported platforms and configuration options here.

Now let's update our Rake tasks to be cloud aware. That way we can specify at run time whether to run things locally or in Sauce.

1
desc 'Run iOS tests'
2
task :ios, :location do |t, args|
3
  location_helper args[:location]
4
  Dir.chdir 'ios'
5
  exec 'rspec'
6
end
7

8
desc 'Run Android tests'
9
task :android, :location do |t, args|
10
  location_helper args[:location]
11
  Dir.chdir 'android'
12
  exec 'rspec'
13
end
14

15
def location_helper(location)
16
  if location != 'sauce'
17
    ENV['SAUCE_USERNAME'], ENV['SAUCE_ACCESS_KEY'] = nil, nil
18
  end
19
end

We've updated our Rake tasks so they can take an argument for the location. We then use this argument value and pass it to location_helper. The location_helper looks at the location value -- if it is not set to 'sauce'then the Sauce credentials get set to nil. This helps us ensure that we really do want to run our tests on Sauce (e.g., we have to specify both the Sauce credentials AND the location).

Now we can launch our tests locally just like before (e.g., rake ios) or in Sauce by specifying it as a location (e.g., rake ios['sauce'])

But in order for the tests to fire in Sauce Labs, we need to specify our credentials somehow. We've opted to keep them out of our Rakefile (and our test code) so that we can maintain future flexibility by not having them hard-coded; which is also more secure since we won't be committing them to our repository.

Specifying Sauce Credentials

There are a few ways we can go about specifying our credentials.

Specify them at run-time

SAUCE_USERNAME=your-username SAUCE_ACCESS_KEY=your-access-key rake ios['sauce']

Export the values into the current command-line session

export SAUCE_USERNAME=your-username export SAUCE_ACCESS_KEY=your-access-key

Set the values in your bash profile (recommended)

# filename: ~/*.bash_profile ... export SAUCE_USERNAME=your-username export SAUCE_ACCESS_KEY=your-access-key

After choosing a method for specifying your credentials, run your tests with one of the Rake task and specify 'sauce' for the location. Then log into your Sauce Account to see the test results and a video of the execution.

Making Your Sauce Runs Descriptive

It's great that our tests are now running in Sauce. But it's tough to sift through the test results since the name and test status are nondescript and all the same. Let's fix that.

Fortunately, we can dynamically set the Sauce Labs job name and test status in our test code. We just need to provide this information before and after our test runs. To do that we'll need to update the RSpec configuration incommon/spec_helper.rb.

1
# filename: common/spec_helper.rb
2

3
...
4
RSpec.configure do |config|
5

6
  config.before(:each) do |example|
7
    $driver.caps[:name] = example.metadata[:full_description] if using_sauce
8
    $driver.start_driver
9
  end
10

11
  config.after(:each) do |example|
12
    if using_sauce
13
      SauceWhisk::Jobs.change_status $driver.driver.session_id, example.exception.nil?
14
    end
15
    driver_quit
16
  end
17

18
end

In before(:each) we update the name attribute of our capabilities (e.g., caps[:name]) with the name of the test. We get this name by tapping into the test's metadata (e.g., example.metadata[:full_description]). And since we only want this to run if we're using Sauce we wrap it in a conditional.

In after(:each) we leverage sauce_whisk to set the job status based on the test result, which we get by checking to see if any exceptions were raised. Again, we only want this to run if we're using Sauce, so we wrap it in a conditional too.

Now if we run our tests in Sauce we will see them execute with the correct name and job status.