Creating new tests for OppiaMobile App

OppiaMobile incorporates several types of tests to make the app stronger.

These types of tests are Local Unit Tests, Instrumented Tests and User Interface Tests.

Local Unit Tests

The Local Unit Tests are tests that run on a local JVM, therefore we do not need a physical device or emulator to run this type of tests, which make the execution time to be shorter.

The main path for these tests is src/test/java. It is mandatory for this type of tests to include the JUnit 4 framework dependency to the app build.gradle file.

testImplementation 'androidx.test.ext:junit:{version}'

The tests methods that we create must have the tag @Test right before the method declaration, and must end with an assertion to check whether the test passes or not. For example:

@Test
public void Storage_correctDownloadPath(){
     …
     assertTrue(downloadPath.isCorrect());
}

Optionally, the tests could provide a preconditions and post conditions blocks

//Preconditions block

@Before
public void setUp() throws Exception{…}

//Post-conditions block

@After
public void tearDown() throws Exception{…}

Instrumented Unit Tests

The Instrumented Unit Tests are test that run on a physical device or emulator. This type of tests is convenient when we need access to instrumentation information (App context for example).

The main path for this tests is src/androidTests/java.

To create and run this test, first we need to install the Android Support Repository from the Android SDK Manager. After that, we need to add some dependencies to the app build.gradle file:

androidTestImplementation 'androidx.annotation:annotation:{version}'
androidTestImplementation 'androidx.test:runner:{version}'
androidTestImplementation 'androidx.test:rules:{version}'
androidTestImplementation 'androidx.test.ext:junit:{version}'
androidTestImplementation 'androidx.test.uiautomator:uiautomator:{version}'
androidTestUtil 'androidx.test:orchestrator:{version}'

In addition, we need to add the default test instrumentation runner to use JUnit 4 test classes, and its parameters:

android {
    defaultConfig {
       testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
       testInstrumentationRunnerArguments clearPackageData: 'true'
       testInstrumentationRunnerArguments coverage: 'true'
       testInstrumentationRunnerArguments coverageFilePath: '/sdcard/coverage/'
    }
   ...

   testOptions {
       execution 'ANDROIDX_TEST_ORCHESTRATOR'
       unitTests.returnDefaultValues = true
       unitTests.includeAndroidResources = true
       animationsDisabled = true
   }
}

When we create a test class, there are some things we have to pay attention.

  • We need to add the @RunWith(AndroidJUnit4.class) tag before the test class definition.

  • We also need to add the @Test tag to all our test methods (as we did in the Local Unit Tests section)

  • The setUp() and tearDown() methods have the same structure as in the Local Unit Tests section.

  • All our tests methods should include the throws Exception line in the method definition.

  • The assertion part is the same as in the Local Unit Tests section.

User Interface Tests

The User Interface Tests are useful to verify that the UI components of the app works correctly and do not provide a poor experience to the final user.

OppiaMobile make use of these tests using the Espresso testing framework, that we might consider it as an Instrumentation-based framework to test UI components.

With Instrumentation-based we mean that it works with the AndroidJUnitRunner test runner (as mention in the previous section).

To use the Espresso library, we need to make sure to follow the same steps described in the previous section (Instrumented Unit Tests) and also we need to add the following dependency to the app build.gradle file:

androidTestImplementation 'androidx.test.espresso:espresso-core:{version}'

  • The Espresso nomenclature is based on three aspects. First we need to find the view we want to test. Next, we have to perform an action over that view. And finally, we need to inspect the result. This is done as follows:

onView(withId(R.id.login_btn))                      //Find the view
    .perform(click());                          //Perform an action
onView(withText(R.string.error_no_username))        //Find the view
    .check(matches(isDisplayed()));       //Inspect the result

Mock Web Server

OppiaMobile makes use of the MockWebServer by okhttp (https://github.com/square/okhttp/tree/master/mockwebserver).

The mock web server is useful to enqueue some responses and in this way testing the client side.

First, we need to add the MockWebServer dependency to our app build.gradle file:

testImplementation 'com.squareup.okhttp3:mockwebserver(insert latest version)’

After that, we are able to create MockWebServer objects. For example:

MockWebServer mockServer = new MockWebServer();

String filename = “responses/response_201_login.json”; //Premade response

mockServer.enqueue(new MockResponse()
    .setResponseCode(201)
    .setBody(getStringFromFile(InstrumentationRegistry
        .getInstrumentation().getContext(), filename)));

mockServer.start();

On the other hand, we need to configure our app to communicate correctly with this mock web server. To achieve that, OppiaMobile uses the class MockApiEndpoint, whose method getFullURL() will give us the correct path on which the mock web server is listening.

Temporary Files and Folders

Junit4 allows us to create temporary files and folders with the guarantee that it will delete all of them when the test finishes, whether the test passes or fails.

The TemporaryFolder object must be created using the @Rule tag.

@Rule
public TemporaryFolder folder = new TemporaryFolder();

//Use
File tempFolder = folder.newFolder(“tempFolder”);
File tempFile = folder.newFile(“tempFile.txt”);

Considerations to create consistent tests

  • When testing screens with ViewPager, if we switch page with smooth scroll, sometimes the test try to handle next event before the transition finishes, so it fails. The quick solution is to add a wait time in the test just after page transition event: Thread.sleep(200) -200ms is enough-. Other solution would be to add an Idling resource but it is not compatible with ViewPager and we’d need to replace with ViewPager2 and its adapter.

  • Espresso waits until AsyncTasks finish if there is a UI change at the end (onPostExecute). If we need to test the result after a AsyncTask without UI change (i.e. only saves data), we need to add a wait time. In slow devices, this could take more than a second so Thread.sleep(3000) should be enough.

  • To test properties which configure UI elements, first add tests to check desired element visibility in UIChecksPropsBased, and then we have to mock the property value in the rest of the tests it is involved. If the property is no mockable (it is not included in the Settings so it has no preference associated), just exit the test (i.e. we cannot test anything related with Settings if MENU_ALLOW_SETTINGS is disabled)

  • Remember to to close soft keyboard after typing text if we need to click any button (keyboard could hide UI elements and make the test fail as they are not clickable). I.e.: onView(withId(R.id.edit_email)).perform(typeText(”any@email.com”), closeSoftKeyboard());

  • If inside a Activity/Fragment there is a repository object with any async process in it, and a ProgressDialog (or similar) is created while the process finishes, there is risk of not closing the ProgressDialog if the repository method (which contains the async process) is mocked and test will fail as there is a not expected view at top. The solution would be to move the code with creates the ProgessDialog inside the repository method or ensure we call the “onComplete” method mocking it in the test with doAnswer(invocationOnMock -> ….)

DEVICE CONFIGURATION BEFORE RUNNING THE TEST:

  • Disable system animations: Settings -> Developer options -> disable: Window animation scale, Transition animation scale and Animator duration scale

  • Disable any auto-fill service: Settings -> Languages and input -> Auto-fill service

  • Some soft keyboards could modify layout behaviour. Ensure the default one is selected (not accessibility or custom): Settings -> Languages and input -> Virtual keyboard

  • Set English language

In addition, if the testing device is physical device:

  • Set Airplane mode to avoid incoming push notifications (system UI notifications could hide part of the screen and make the tests fail)

  • Disable screen rotation