20 Jul 2012 10:32
Sonar Android support
Stéphane Nicolas <snicolas@...>
2012-07-20 08:32:37 GMT
2012-07-20 08:32:37 GMT
50, Avenue des Champs-Elysées
Mob : (33) 126.96.36.199.09
I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.
I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude :
Limitations / Differences between Android and Standard Java projects :
- Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
- The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests :
- After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version 2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller).
- Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
- Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed.
- With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised :
- If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
- The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
- As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
- Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
- Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric.
- I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
- An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
- Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
- Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
- Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
- Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.
To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.
Thanks for reading,--
Développeur & Consultant Android / Java
50, Avenue des Champs-Elysées
Mob : (33) 188.8.131.52.09