Testing code

Students who write bug free code can leave the class now -- they get an A. Those students should come see me during my office hours.

You have come across unit testing before in CS50. You might think that great coders write great code and that is it; they never need to test their code -- that is, systematically determine no bugs lurk at the edges of their code or any where in their objects. Wrong! Great coders are great testers.

We have written quite a lot of Android code at this point. You've used Log and the debugger but we've never tested our code.

But how do you test your app? Think about MyRuns. It has many classes. It has a UI, activities, services, loaders, database, many fragments, etc.

How do we know the UI does not leak data or that the SQLite objects manage the data in the database in consistent manner. We may have used the app, entered various data and tracked a run but how do we really know there are no bugs in the app. We don't. I'm sure they exist.

In industry you will sometimes write test cases before you write the actual code. No matter what you will always test the code you write. So if you plan to write code you better get use to the idea that you need to know how to test code.

As an aside, my very first software job was unit testing of major parts of a kernel OS. I tested the code written by a brilliant hacker. I spent one year of my sad life as tester trying to prove there were bugs. I never found a single bug. But that's another story.

What this lecture will teach you

Resources

Testing framework

The Android testing framework provides an tools that help you test every aspect of your application at every level from unit (method, object) to the complete app framework.

Android test suites are based on JUnit. You can use plain JUnit to test a class that does not call the Android API, or Android's JUnit extensions to test Android components.

The Android JUnit extensions provide - specific test case classes. These classes provide helper methods for creating mock objects and methods that help you control the lifecycle of a component. You can test the lifecycle of various components, such as, fragments, activities, services, etc. You can test different configurations. It's cool.

Problem is that there isn't a lot of details for testing with the relatively new Android Studio. Most web based information relates to testing using the Eclipse IDE which is quite different from Studio.

Test suites are contained in test packages that are similar to main application packages, so you do not need to learn a new set of tools or techniques for designing and building tests. You can do that right in the Android Studio.

The SDK tools for building and tests are available in Android Studio, and also in command-line (we will not use command line). These tools get information from the project (for the application) under test and uses this information to automatically create the build files, manifest file, and directory structure for the test package.

The SDK also uses monkeyrunner, an API for testing devices with Python programs, and UI/Application Exerciser Monkey, a command-line tool for stress-testing UIs by sending pseudo-random events to a device.

Create your test class

You can create a test project anywhere in your file system, but the best approach is to add the test project so that its root directory tests/ is at the same level as the src/ directory of the main application's project. This helps you find the tests associated with particular application. For example, in the MyRuns1 example, right click the java package and create a package called "tests". And we create a Java class in the test package.

The Android testing API is based on the JUnit API and extended with a instrumentation framework and Android-specific testing classes. You can use the JUnit Assert class to display test results. The assert method compares values you expect from a test to the actual results. If the value is as expected then that particular test passes. If it is not the value you expect then the test throws an exception -- that is, if the comparison fails. Android also provides a class of assertions that extend the possible types of comparisons. It also provides another class of assertions for testing the UI.

For example, we would likely want to test the UI code. And we would also likely want to test all the objects, all the paths through the app code and each method in every object. You can see as you app grows in size and complexity your test cases and test code will grow. That's to be expected. But you should not feel overwhelmed because you can test at the method level, object level, app level. I like to start bottom up in case of the system components but probably top down for the UI.

You must use Android’s instrumented test runner – AndroidJUnitRunner – to run your test case classes. For this, you need to first import support libraries for testing. You can include these in your gradle file.

The build.gradle file for the app should also have a testInstrumentationRunner specification of "androidx.test.runner.AndroidJUnitRunner".

Then you are ready to start writing test cases. Individual tests are annotated with the word ‘Test’. Here is an example:

@RunWith(AndroidJUnit4.class)
public class ProfileActivityTest {

    @Test
    public void testExample() {
        int expect = 1;
        int actual = 2;
        assertEquals(expect, actual);
    }
}

How to run your test

First you need to set up the configurations by choosing "Run" -> "Edit Configurations". Then click the "+" button from the upper left corner to add an Android Tests configuration, as below. Let's walk you through the process.

Type the name of the test, select the current module ("app") as the module, select the “All in Package” option and navigate to your “tests” folder you created. The configuration should look like the following:

Apply the changes and close the dialog. You should now see your test cases as a runnable project configuration in the bar across the top of your Android Studio instance.

Choose the test instance, then run it. You should now see the failed test result because the expected value is different to the actual value.

Testing MyRuns1

Since we are testing an activity, first we need to specifiy an ActivityTestRule and then instantiate the class name that we want to test.

import androidx.test.ext.junit.runners.AndroidJUnit4;
import androidx.test.rule.ActivityTestRule;

@RunWith(AndroidJUnit4.class)
public class ProfileActivityTest {

    @Rule
    public ActivityTestRule<ProfileActivity> mActivityRule = new ActivityTestRule(ProfileActivity.class);
}

The ActivityTestRule class provides functional testing of a single activity. The Activity under test will be launched before each test annotated with Test. You will then be able to manipulate your Activity directly.

Myruns1 allows the user to enter and save: name, email, phone etc. Next, we will create a test to make sure that value of name entered persists across activity restarts. So if you flipped the orientation your input values e.g., name, is saved corrected and reloaded when you change the configuration from portrait to landscape. Clearly, we might have hand tested this but we now get the code to test this systematically. Remember, if you add code to a class that say you have developed. You rerun all the tests to confirm that the new code has not broken the old code - there can be crosstalk between code.

Inorder to get the views in Profile activity, we need to import the R values -- R.java

import edu.dartmouth.cs.myruns.R;

Next, we create a test method in the test class. Again, all test methods must be annotated as “Test", otherwise Android Studio will not detect them as tests.

        final String TEST_NAME_VALUE = "test_name";

We can launch the activity using getActivity() on the activity rule, and get the views in the XML file using the same approach as in the activity class.

      // Launch the activity
      ProfileActivity activity = mActivityRule.getActivity();

      // Get name edit text and save button
      final EditText text = (EditText) activity.findViewById(R.id.editName);
      final Button save = (Button) activity.findViewById(R.id.btnSave);

Next, we need to set the name value to the edit text (i.e., EditText) and click the save button. Because we are manipulating the UI, the execution must be performed on a UI thread. Makes sense.

An application's activities run on the application's UI thread. Once the UI is instantiated, for example in the activity's onCreate() method, then all interactions with the UI must run in the UI thread. When you run the application normally, it has access to the thread and does not have to do anything special.

This changes when you run tests against the application. With instrumentation-based classes, you can invoke methods against the UI of the application under test. The other test classes do not allow this. To run an entire test method on the UI thread, you can annotate the thread with @UIThreadTest. Notice that this will run all of the method statements on the UI thread. Methods that do not interact with the UI are not allowed; for example, you can't invoke Instrumentation.waitForIdleSync().

To run a subset of a test method on the UI thread, create an anonymous class of type Runnable, put the statements you want in the run() method, and instantiate a new instance of the class as a parameter to the method appActivity.runOnUiThread(), where appActivity is the instance of the application you are testing.

        activity.runOnUiThread(new Runnable() {
            @Override
            public void run() {
                // Attempts to manipulate the UI must be performed on a UI thread.
                // Calling this outside runOnUiThread() will cause an exception.
                //
                // You could also use @UiThreadTest, but activity lifecycle methods
                // cannot be called if this annotation is used.
                text.requestFocus();
                //set the name to the edit text
                text.setText(TEST_NAME_VALUE);
                //click the save button
                save.performClick();
            }
        });

Calling .setText() or performClick() outside runOnUiThread() will cause an exception. Next we close the activity and relaunch it, get the name value from the name edit text, check if it is the same as the one we set before.

        // Close the activity
        activity.finish();
        mActivityRule.launchActivity(null);  // Required to force creation of a new activity

        // Relaunch the activity
        activity = mActivityRule.getActivity();

        // Verify that the name was saved at the name edit text
        final EditText name2 = (EditText) activity.findViewById(R.id.editName);
        String currentName = name2.getText().toString();
        assertEquals(TEST_NAME_VALUE, currentName);

If your program is correct, you should see it passes the test. Otherwise you should verify the correctness of your implementation.

Change in orientation of the phone

As mentioned in the topic of "What to test" on Android official website, testing change in orientation is one of the important test cases that you must consider when testing your app.

For devices that support multiple orientations, Android detects a change in orientation when the user turns the device so that the display is "landscape" (long edge is horizontal) instead of "portrait" (long edge is vertical).

When Android detects a change in orientation, its default behavior is to destroy and then re-start the foreground Activity. We have discussed how the activity and fragment lifecycle respond to such configurations changes multiple times in class. OK, you should consider testing the following after changing the orientation (i.e., configuration):

Is the screen re-drawn correctly? Any custom UI code you have should handle changes in the orientation? Does the application maintain its state? The Activity should not lose anything that the user has already entered into the UI. The application should not "forget" its place in the current transaction.

In Myruns1, we need to save the image temporarily inside onSaveInstanceState() when the screen roates thus the image persists after screen rotation. Now we test if such feature is implemented correctly.

In order to set a test image, we first start the activity, get an image from drawable folder and convert the bimap into byte array.

       // Launch the activity
       ProfileActivity activity = mActivityRule.getActivity();

       // Define a test bitmap
       final Bitmap TEST_BITMAP = BitmapFactory.decodeResource(activity.getResources(),R.drawable.blue_pushpin);
                
       // Convert bitmap to byte array
       ByteArrayOutputStream bos = new ByteArrayOutputStream();
       TEST_BITMAP.compress(Bitmap.CompressFormat.PNG, 100, bos);
       final byte[] TEST_BITMAP_VALUE = bos.toByteArray();

        final ImageView mImageView = (ImageView) activity.findViewById(R.id.imageProfile);
        activity.runOnUiThread(new Runnable() {
            @Override
            public void run() {
                //set the test bitmap to the image view
                mImageView.setImageBitmap(TEST_BITMAP);
            }
        });

We can rotate the screen by using setRequestedOrientation(). However, it creates a new instance of the activity when we rotate the screen, thus, we need to use Activity monitor to track the activity and update the current activity. Instrumentation.ActivityMonitor provides information about a particular kind of Intent that is being monitored. An instance of this class is added to the current instrumentation through addMonitor(Instrumentation.ActivityMonitor); after being added, when a new activity is being started the monitor will be checked and, if matching, its hit count updated and (optionally) the call stopped and a canned result returned.

The rotation test is required to open your phone screen, otherwise the test will hang.

        Instrumentation.ActivityMonitor monitor = new Instrumentation.ActivityMonitor(ProfileActivity.class.getName(), null, false);
        getInstrumentation().addMonitor(monitor);
        // Rotate the screen
        activity.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
        getInstrumentation().waitForIdleSync();
        // Updates current activity
        ProfileActivity activity_updated = (ProfileActivity) getInstrumentation().waitForMonitor(monitor);

Now the “activity_updated” variable is the rotated activity. You can find the bitmap value on the image view and compare it to our test bitmap.

        // Convert bitmap to byte array
        bos = new ByteArrayOutputStream();
        currentBitMap.compress(Bitmap.CompressFormat.PNG, 100, bos);
        byte[] currentBitmapValue = bos.toByteArray();

        // Check if these two bitmaps have the same byte values.
        // If the program executes correctly, they should be the same
         assertArrayEquals(TEST_BITMAP_VALUE, currentBitmapValue);

CodeItFour: testing and tracking down a bug

OK. For this week's codeIt you have to complete the testing of the UI for the profile in MyRun3-- which requires you to test the UI. After you have done that you will test the database helper object in myRuns3. Here you are not testing the UI but all the methods in the class.

We will provide you with the solution code for MyRuns3 so you all have the same code. This is an individual exercise and not with a partner.

The final twist is that we have introduced a bug in one part of the code you will test. So testing should find the bug. Once you find the bug fix it in your code. So then all tests will pass.

Lecture 27 - Testing