Android Gradle Jacoco: Offline Instrumentation For Integration Tests

we are building an Android app which is tested by using Appium. Now I would like to see the test coverage of our Appium tests. I think this is possible, because Jacoco supports off

Solution 1:

Finally I managed it to get it working and I want to share the solution with you:

enable instrumentation for your buildType and configure SonarQube accordingly e.g.

...
apply plugin: 'jacoco'
...

android {
    ...
    productFlavors {
        acceptance {
            applicationId packageName + ".acceptance"
            buildTypes {
                debug {
                    testCoverageEnabled true
                }
            }
        }
    }
}


sonarRunner {
    sonarProperties {
        property"sonar.host.url", "..."property"sonar.jdbc.url", sonarDatabaseUrl
        property"sonar.jdbc.driverClassName", sonarDatabaseDriverClassName
        property"sonar.jdbc.username", sonarDatabaseUsername
        property"sonar.jdbc.password", sonarDatabasePassword

        property"sonar.sourceEncoding", "UTF-8"property"sonar.sources", "src/main"property"sonar.tests", "src/test"property"sonar.inclusions", "**/*.java,**/*.xml"property"sonar.import_unknown_files", "true"property"sonar.java.binaries", "build/intermediates/classes/acceptance/debug"property"sonar.junit.reportsPath", "build/test-results/acceptanceDebug"property"sonar.android.lint.report", "build/outputs/lint-results.xml"property"sonar.java.coveragePlugin", "jacoco"property"sonar.jacoco.reportPath", "build/jacoco/testAcceptanceDebugUnitTest.exec"
        // see steps below on how to get that file:
        property"sonar.jacoco.itReportPath", "build/jacoco/jacoco-it.exec"property"sonar.projectKey", projectKey
        property"sonar.projectName", projectName
        property"sonar.projectVersion", appVersionName
    }
}

add the following to your AndroidManifest.xml

<receiverandroid:name=".util.CoverageDataDumper"tools:ignore="ExportedReceiver"><intent-filter><actionandroid:name="org.example.DUMP_COVERAGE_DATA"/></intent-filter></receiver>

CoverageDataDumper should look like that:

publicclassCoverageDataDumperextendsBroadcastReceiver {
   privatestaticfinalLoggerLOG= LoggerFactory.getLogger( CoverageDataDumper.class );

   @OverridepublicvoidonReceive( Context context, Intent intent ) {
      try {
         Class
            .forName( "com.vladium.emma.rt.RT" )
            .getMethod( "dumpCoverageData", File.class, boolean.class, boolean.class )
            .invoke( null,
               newFile( App.getContext().getExternalFilesDir( null ) + "/coverage.ec" ),
               true, // mergefalse// stopDataCollection
            );
      }
      catch ( Exception e ) {
         LOG.error( "Error when writing coverage data", e );
      }
   }
}

Then run your Appium test cases with the acceptance flavor app (with instrumented classes). Before you call "Reset App" or "Close Application" make sure to call the following methods (just a draft, but I think you get the idea):

// intent is "org.example.DUMP_COVERAGE_DATA"publicvoidendTestCoverage(String intent ) {
  if ( driver instanceofAndroidDriver ) {
     ((AndroidDriver) driver).endTestCoverage( intent, "" );
  }
}
publicvoidpullCoverageData(String outputPath ) {
  String coverageFilePath = (String) appiumDriver.getCapabilities().getCapability( "coverageFilePath" );
  if ( coverageFilePath != null ) {
     byte[] log = appiumDriver.pullFile( coverageFilePath );
     MobileAppLog.writeLog( newFile( outputPath ), log );
  }
  else {
     thrownewAppiumLibraryNonFatalException(
        "Tried to pull the coverage data, but the coverageFilePath wasn't specified." );
  }
}

outputPath could be for example: /sdcard/Android/data/org.example.acceptance/files/coverage.ec

Now the Jacoco data is written to the Smartphone. Next we need to download that file. You can use

appiumDriver.pullFile( logFilePath );

Now you need to copy the file "jacoco-it.exec" (which should always be appended when you pull the file) into build/jacoco/jacoco-it.exec see gradle.build above and run

gradlew sonarRunner

In SonarQube add the Integration Test Coverage Widget and you should see now some values...

Unfortunately code coverage won't work if you are using retrolambda (as we do). Retrolambda will generate anonymous classes which are not part of the source files - so SonarQube cannot match them correctly and displays a much lower code coverage than it actually is. If someone finds a solution for that, I would be very happy :-)

Solution 2:

I solved this problem by adding broadcast receiver to the application you test! (you can add the receiver only to debug folder cause no need it for to exist in main source)

publicclassCoverageReceiverextendsBroadcastReceiver{
    privatestaticfinalString EXEC_FILE_PATH = "/mnt/sdcard/coverage.exec";
    privatestaticfinalString TAG = "CoverageJacoco";
    privatestaticfinalString BROADCAST_RECEIVED_MESSAGE = "EndJacocoBroadcast broadcast received!";
    privatestaticfinalString EMMA_CLASS = "com.vladium.emma.rt.RT";
    privatestaticfinalString EMMA_DUMP_METHOD = "dumpCoverageData";
@Override
publicvoid onReceive(Context context, Intent intent) {
    try {
        Log.d(TAG, BROADCAST_RECEIVED_MESSAGE);
        Class.forName(EMMA_CLASS)
                .getMethod(EMMA_DUMP_METHOD, File.class, boolean.class,
                        boolean.class)
                .invoke(null, new File(EXEC_FILE_PATH), true,
                        false);
    } catch (Exception e) {
        Log.d(TAG, e.getMessage());
    }
}
}

In manefist add (you can add this debug folder so it won't exist in main source)

<?xml version="1.0" encoding="utf-8"?><manifestxmlns:android="http://schemas.android.com/apk/res/android" ><uses-permissionandroid:name="android.permission.WRITE_EXTERNAL_STORAGE" /><application><receiverandroid:name=".CoverageReceiver"><intent-filter><actionandroid:name="com.example.action" /></intent-filter></receiver></application>

In the build.gradle of the application I added

apply plugin: 'jacoco'

jacoco {
    toolVersion = "0.7.4+"
}

model {
    android {
        compileSdkVersion 23
        buildToolsVersion "23.0.2"
    defaultConfig {
        applicationId "com.example.app"
        minSdkVersion.apiLevel 23
        targetSdkVersion.apiLevel 23
        versionCode 12
        versionName "1.11"

    }
    buildTypes {

        debug {
            testCoverageEnabled true

        }
    }

you build your application as debug, than install and run it.

send broadcast through ADB "adb shell am broadcast -a com.example.action" to create coverage.exec pull coverage from device - adb pull /mnt/sdcard/coverage.exec

after you run this you need to create the coverage from the file

   **
 * This task is used to create a code coverage report via the Jcoco tool.
 */
task jacocoTestReport(type: JacocoReport) {
    def coverageSourceDirs = [
            'src/main/java',               
    ]
    group = "Reporting"
    description = "Generates Jacoco coverage reports"
    reports {
        csv.enabled false
        xml{
            enabled = true
            destination "${buildDir}/jacoco/jacoco.xml"
        }
        html{
            enabled true
            destination "${buildDir}/jacocoHtml"
        }
    }
    classDirectories = fileTree(
            dir: 'build/intermediates/classes',
            excludes: ['**/R.class',
                       '**/R$*.class',
                       '**/BuildConfig.*',
                       '**/Manifest*.*',
                       '**/*Activity*.*',
                       '**/*Fragment*.*'
            ]
    )
    sourceDirectories = files(coverageSourceDirs)
    executionData = files('build/coverage.exec')
}

this task is one way to create coverage files in coverageSourceDirs add all the locations of your applicaiton source code, so it will know which code to take and create coverage based on them executionData is the location where you put the coverage.exec you pulled from the device

Run the task the files will created for html and xml you can also add csv (notice it will be create in the build folder of the application)!

Need to know, you must run the task against the same code you built your application debug version