Stéphane Nicolas | 20 Jul 10:32 2012

Sonar Android support

Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................

Evgeny Mandrikov | 23 Jul 14:37 2012
Picon

Re: Sonar Android support

Hi,


First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_
Stéphane Nicolas | 25 Jul 20:44 2012

Re: Sonar Android support

Hello Evgeny,


I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................

Evgeny Mandrikov | 26 Jul 06:41 2012
Picon

Re: Sonar Android support

Hi,


See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_
Stéphane Nicolas | 29 Jul 07:54 2012

Re: Sonar Android support

Hello Evgeny,


when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane

2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................

Stéphane Nicolas | 3 Jan 17:17 2013

Re: Sonar Android support

Hi all,

 Hi Evgeny,

I just opened a maven repository for the sample android project we had been talking about a few months ago (in this thread).
So, your idea was to setup a sample project + tests + test results and code coverage (using emma) : this is the project I created on github.


The project can run easily inside an open emulator or on a plugged android device. Tell me if you want me to add anything or need any advice.

At OCTO France, the place where I work we are 2 devs very motivated to bring Android to Sonar and the other around, and our colleagues in Switzerland are motivated to achieve the same for iOS...

For us, the next step is to create a lint plugin for sonar. For that we needed a maven mojo to invoke lint and I commited a pull request to the Android Maven Plugin to run lint : https://github.com/jayway/maven-android-plugin/pull/168

So basically, we are almost ready to get started for the lint sonar plugin. We will need some advices to get started.
Does the sample match your needs to start a sonar android project ?

Greetings,
 Stéphane



2012/7/29 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hello Evgeny,

when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane


2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................
Evgeny Mandrikov | 3 Jan 17:25 2013
Picon

Re: Sonar Android support

Hi,

In fact I waited a really long time and yesterday created such example by myself : https://github.com/SonarSource/sonar-examples/tree/master/projects/android ;) which I suppose should look pretty similar to yours.

Regarding lint integration - there is an info in Confluence ( http://docs.codehaus.org/display/SONAR/Coding+a+Plugin ) and tons of open-source plugins. So which additional advices do you expect?


On Thu, Jan 3, 2013 at 5:17 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi all,
 Hi Evgeny,

I just opened a maven repository for the sample android project we had been talking about a few months ago (in this thread).
So, your idea was to setup a sample project + tests + test results and code coverage (using emma) : this is the project I created on github.


The project can run easily inside an open emulator or on a plugged android device. Tell me if you want me to add anything or need any advice.

At OCTO France, the place where I work we are 2 devs very motivated to bring Android to Sonar and the other around, and our colleagues in Switzerland are motivated to achieve the same for iOS...

For us, the next step is to create a lint plugin for sonar. For that we needed a maven mojo to invoke lint and I commited a pull request to the Android Maven Plugin to run lint : https://github.com/jayway/maven-android-plugin/pull/168

So basically, we are almost ready to get started for the lint sonar plugin. We will need some advices to get started.
Does the sample match your needs to start a sonar android project ?

Greetings,
 Stéphane



2012/7/29 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hello Evgeny,

when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane


2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_
Stéphane Nicolas | 4 Jan 06:43 2013

Re: Sonar Android support

Hi Evgeny,


I am glad to hear that we are both motivated and had the same idea almost at the same time.

Actually your sample has a parent pom but mine goes a bit further than yours (for instance I download the code coverage results from the device/emulator). I would be pleased to maintain the samples for android for now as we are gonna work on them.

I added you to our repo has collaborator. Do you mind if we go ahead with our sample ? I can add a parent pom if your feel more confortable with it, but I would vote against sharing any dependency management so that the sample and tests are real samples, more easy to copy paste for developers.

So my question is, more precisely : 
* do we need to add, as in sonar-java, some kind of central sonar-android project to get android related stuff centralized ? Would it need to check an android project foot print ?
* if I understand well, the only thing we would have to do for a sonar lint plugin is a new sonar sensor that will parse lint results and add the issues to the sonar database, right ? 
* will we need to convert the lint error levels to sonar error levels ?

Stéphane

2013/1/3 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

In fact I waited a really long time and yesterday created such example by myself : https://github.com/SonarSource/sonar-examples/tree/master/projects/android ;) which I suppose should look pretty similar to yours.

Regarding lint integration - there is an info in Confluence ( http://docs.codehaus.org/display/SONAR/Coding+a+Plugin ) and tons of open-source plugins. So which additional advices do you expect?


On Thu, Jan 3, 2013 at 5:17 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi all,
 Hi Evgeny,

I just opened a maven repository for the sample android project we had been talking about a few months ago (in this thread).
So, your idea was to setup a sample project + tests + test results and code coverage (using emma) : this is the project I created on github.


The project can run easily inside an open emulator or on a plugged android device. Tell me if you want me to add anything or need any advice.

At OCTO France, the place where I work we are 2 devs very motivated to bring Android to Sonar and the other around, and our colleagues in Switzerland are motivated to achieve the same for iOS...

For us, the next step is to create a lint plugin for sonar. For that we needed a maven mojo to invoke lint and I commited a pull request to the Android Maven Plugin to run lint : https://github.com/jayway/maven-android-plugin/pull/168

So basically, we are almost ready to get started for the lint sonar plugin. We will need some advices to get started.
Does the sample match your needs to start a sonar android project ?

Greetings,
 Stéphane



2012/7/29 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hello Evgeny,

when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane


2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................
Evgeny Mandrikov | 4 Jan 10:09 2013
Picon

Re: Sonar Android support

Hi,

See my comments below:


On Fri, Jan 4, 2013 at 6:43 AM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi Evgeny,

I am glad to hear that we are both motivated and had the same idea almost at the same time.

Actually your sample has a parent pom but mine goes a bit further than yours (for instance I download the code coverage results from the device/emulator).
I would be pleased to maintain the samples for android for now as we are gonna work on them.

I added you to our repo has collaborator. Do you mind if we go ahead with our sample ? I can add a parent pom if your feel more confortable with it, but I would vote against sharing any dependency management so that the sample and tests are real samples, more easy to copy paste for developers.

First of all - don't treat this as a competition.
And be careful, because my example does exactly the same : it pulls coverage results from device/emulator - see https://github.com/SonarSource/sonar-examples/blob/master/projects/android/app.tests/pom.xml#L73
Moreover - it uses newer version of Emma, so no need to hack Sonar Emma Plugin - it works out-of-the-box.
Parent POM is a convenient way to have a single reactor. And I don't see problems for developers to copy-paste it - anyway they would have two modules (app and tests).
And excuse me, but I won't contribute into your repository, because http://github.com/SonarSource/sonar-examples is an official examples from SonarSource.

So my question is, more precisely : 
* do we need to add, as in sonar-java, some kind of central sonar-android project to get android related stuff centralized ? Would it need to check an android project foot print ?

For me Android project is a just another Java project, so no.
  
* if I understand well, the only thing we would have to do for a sonar lint plugin is a new sonar sensor that will parse lint results and add the issues to the sonar database, right ? 

Yes.
 
* will we need to convert the lint error levels to sonar error levels ?

Yes.
 
Stéphane

2013/1/3 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

In fact I waited a really long time and yesterday created such example by myself : https://github.com/SonarSource/sonar-examples/tree/master/projects/android ;) which I suppose should look pretty similar to yours.

Regarding lint integration - there is an info in Confluence ( http://docs.codehaus.org/display/SONAR/Coding+a+Plugin ) and tons of open-source plugins. So which additional advices do you expect?


On Thu, Jan 3, 2013 at 5:17 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi all,
 Hi Evgeny,

I just opened a maven repository for the sample android project we had been talking about a few months ago (in this thread).
So, your idea was to setup a sample project + tests + test results and code coverage (using emma) : this is the project I created on github.


The project can run easily inside an open emulator or on a plugged android device. Tell me if you want me to add anything or need any advice.

At OCTO France, the place where I work we are 2 devs very motivated to bring Android to Sonar and the other around, and our colleagues in Switzerland are motivated to achieve the same for iOS...

For us, the next step is to create a lint plugin for sonar. For that we needed a maven mojo to invoke lint and I commited a pull request to the Android Maven Plugin to run lint : https://github.com/jayway/maven-android-plugin/pull/168

So basically, we are almost ready to get started for the lint sonar plugin. We will need some advices to get started.
Does the sample match your needs to start a sonar android project ?

Greetings,
 Stéphane



2012/7/29 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hello Evgeny,

when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane


2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_
Stéphane Nicolas | 4 Jan 16:28 2013

Re: Sonar Android support

Hi Evgeny,


fine, I didn't see that you were pulling emma files from device or emulator. Nice you did.

I don't see this as a competition, but will drop out my repo and will fork yours as needed, except if I get access to yours.

So if we treat Android project as a kind of Java projects, we will start working on a standalone plugin for sonar to integrate lint.
We will come back to you very soon. 

Thanks for your fast answer,
Stéphane

2013/1/4 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

See my comments below:


On Fri, Jan 4, 2013 at 6:43 AM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi Evgeny,

I am glad to hear that we are both motivated and had the same idea almost at the same time.

Actually your sample has a parent pom but mine goes a bit further than yours (for instance I download the code coverage results from the device/emulator).
I would be pleased to maintain the samples for android for now as we are gonna work on them.

I added you to our repo has collaborator. Do you mind if we go ahead with our sample ? I can add a parent pom if your feel more confortable with it, but I would vote against sharing any dependency management so that the sample and tests are real samples, more easy to copy paste for developers.

First of all - don't treat this as a competition.
And be careful, because my example does exactly the same : it pulls coverage results from device/emulator - see https://github.com/SonarSource/sonar-examples/blob/master/projects/android/app.tests/pom.xml#L73
Moreover - it uses newer version of Emma, so no need to hack Sonar Emma Plugin - it works out-of-the-box.
Parent POM is a convenient way to have a single reactor. And I don't see problems for developers to copy-paste it - anyway they would have two modules (app and tests).
And excuse me, but I won't contribute into your repository, because http://github.com/SonarSource/sonar-examples is an official examples from SonarSource.

So my question is, more precisely : 
* do we need to add, as in sonar-java, some kind of central sonar-android project to get android related stuff centralized ? Would it need to check an android project foot print ?

For me Android project is a just another Java project, so no.
  
* if I understand well, the only thing we would have to do for a sonar lint plugin is a new sonar sensor that will parse lint results and add the issues to the sonar database, right ? 

Yes.
 
* will we need to convert the lint error levels to sonar error levels ?

Yes.
 
Stéphane

2013/1/3 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

In fact I waited a really long time and yesterday created such example by myself : https://github.com/SonarSource/sonar-examples/tree/master/projects/android ;) which I suppose should look pretty similar to yours.

Regarding lint integration - there is an info in Confluence ( http://docs.codehaus.org/display/SONAR/Coding+a+Plugin ) and tons of open-source plugins. So which additional advices do you expect?


On Thu, Jan 3, 2013 at 5:17 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi all,
 Hi Evgeny,

I just opened a maven repository for the sample android project we had been talking about a few months ago (in this thread).
So, your idea was to setup a sample project + tests + test results and code coverage (using emma) : this is the project I created on github.


The project can run easily inside an open emulator or on a plugged android device. Tell me if you want me to add anything or need any advice.

At OCTO France, the place where I work we are 2 devs very motivated to bring Android to Sonar and the other around, and our colleagues in Switzerland are motivated to achieve the same for iOS...

For us, the next step is to create a lint plugin for sonar. For that we needed a maven mojo to invoke lint and I commited a pull request to the Android Maven Plugin to run lint : https://github.com/jayway/maven-android-plugin/pull/168

So basically, we are almost ready to get started for the lint sonar plugin. We will need some advices to get started.
Does the sample match your needs to start a sonar android project ?

Greetings,
 Stéphane



2012/7/29 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hello Evgeny,

when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane


2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................
Stéphane Nicolas | 18 Feb 00:14 2013

Re: Sonar Android support

We finally find out some time to initiate a sonar lint plugin repo on git hub.


This is our first sonar plugin, and up to now, we didn't have much time, but still, the project is up. 
Contributions are more than welcome :)

Stéphane

2013/1/4 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hi Evgeny,

fine, I didn't see that you were pulling emma files from device or emulator. Nice you did.

I don't see this as a competition, but will drop out my repo and will fork yours as needed, except if I get access to yours.

So if we treat Android project as a kind of Java projects, we will start working on a standalone plugin for sonar to integrate lint.
We will come back to you very soon. 

Thanks for your fast answer,
Stéphane


2013/1/4 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:


On Fri, Jan 4, 2013 at 6:43 AM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi Evgeny,

I am glad to hear that we are both motivated and had the same idea almost at the same time.

Actually your sample has a parent pom but mine goes a bit further than yours (for instance I download the code coverage results from the device/emulator).
I would be pleased to maintain the samples for android for now as we are gonna work on them.

I added you to our repo has collaborator. Do you mind if we go ahead with our sample ? I can add a parent pom if your feel more confortable with it, but I would vote against sharing any dependency management so that the sample and tests are real samples, more easy to copy paste for developers.

First of all - don't treat this as a competition.
And be careful, because my example does exactly the same : it pulls coverage results from device/emulator - see https://github.com/SonarSource/sonar-examples/blob/master/projects/android/app.tests/pom.xml#L73
Moreover - it uses newer version of Emma, so no need to hack Sonar Emma Plugin - it works out-of-the-box.
Parent POM is a convenient way to have a single reactor. And I don't see problems for developers to copy-paste it - anyway they would have two modules (app and tests).
And excuse me, but I won't contribute into your repository, because http://github.com/SonarSource/sonar-examples is an official examples from SonarSource.

So my question is, more precisely : 
* do we need to add, as in sonar-java, some kind of central sonar-android project to get android related stuff centralized ? Would it need to check an android project foot print ?

For me Android project is a just another Java project, so no.
  
* if I understand well, the only thing we would have to do for a sonar lint plugin is a new sonar sensor that will parse lint results and add the issues to the sonar database, right ? 

Yes.
 
* will we need to convert the lint error levels to sonar error levels ?

Yes.
 
Stéphane

2013/1/3 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

In fact I waited a really long time and yesterday created such example by myself : https://github.com/SonarSource/sonar-examples/tree/master/projects/android ;) which I suppose should look pretty similar to yours.

Regarding lint integration - there is an info in Confluence ( http://docs.codehaus.org/display/SONAR/Coding+a+Plugin ) and tons of open-source plugins. So which additional advices do you expect?


On Thu, Jan 3, 2013 at 5:17 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hi all,
 Hi Evgeny,

I just opened a maven repository for the sample android project we had been talking about a few months ago (in this thread).
So, your idea was to setup a sample project + tests + test results and code coverage (using emma) : this is the project I created on github.


The project can run easily inside an open emulator or on a plugged android device. Tell me if you want me to add anything or need any advice.

At OCTO France, the place where I work we are 2 devs very motivated to bring Android to Sonar and the other around, and our colleagues in Switzerland are motivated to achieve the same for iOS...

For us, the next step is to create a lint plugin for sonar. For that we needed a maven mojo to invoke lint and I commited a pull request to the Android Maven Plugin to run lint : https://github.com/jayway/maven-android-plugin/pull/168

So basically, we are almost ready to get started for the lint sonar plugin. We will need some advices to get started.
Does the sample match your needs to start a sonar android project ?

Greetings,
 Stéphane



2012/7/29 Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org>
Hello Evgeny,

when you say : I understand this (about emma). My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?

Emma reports (if you use the sonar emma plugin + bug fix for emma version transcoding) can be integrated into sonar and that works. 
But as I told you, more than often they are not true unit tests, but integration tests. Thus they should normally appear as integration test reports, and that is not possible).

I will have a look at a way to plugin sonar and lint, by writing a lint sensor.
I will provide a small example project. What you describe is merely achieved just by creating an android project using acquinet android-with-tests archetype.
But yes, there will be some additional installation steps, as you mentionned it, it will need an android sdk download. Can I simply put that in README.txt file ?

Regards,
 Stéphane


2012/7/26 Evgeny Mandrikov <mandrikov <at> gmail.com>
Hi,

See my comments below:

On Thu, Jul 26, 2012 at 12:44 AM, Stéphane Nicolas <snicolas <at> octo.com> wrote:
Hello Evgeny,

I am glad to talk with author of the emma plugin for sonar, I have used your code to build the undercover sonar plugin.

About the idea to replace emma by Jacoco :  I second you and agree with the idea to make jacoco working for android integration tests. That would push the platform forward and make Android more standard compliant within the actual java world.

As was stated before - examples of projects will help to force this process. 
 
About the link between emma and integration tests : on android, usual tests (using the android framework) are executed on a device / emulator. Not only because of the difference between dalvik and a standard jvm, but also because most classes of the android system (including what developpers actually do when they develop for android) is tied to an execution context provided by the device / emulator. Thus there are just a few tests that you can write an android that can be called unit tests, there will most of the time be a bunch of (transparent) dependencies in android classes : those tests are integration tests.

I understand this. My question was: what prevents from showing coverage for such tests in Sonar as a coverage (not as integration tests coverage)?
 
To get real unit tests, an alternative approach is to stub / mock android classes, and this can be done using roboelectric and powermockito, with undercover as the unit test code coverage engine. A sonar plugin for undercover is available (but still in its infancy, see above).

About the new version of emma using  SONARPLUGINS-1356 : yes this is absolutely needed for android. Except if we can make things work with jacoco.. :)

About lint : ok, show me how to do a sonar plugin for it, where to start and I would be happy to start such an effort if it's not too big or even if it's big but I hope other coders will follow.

Well, you can take a look on our documentation and existing plugins. Typical strategy for third-party tools is to implement sensor, which will:
  • execute tool to generate xml report
  • parse report to inject data into Sonar
 
About android samples : I would be glad to. And I believe this could be done pretty fast. What exactly should I put inside ? An app, its test app, some maven and ant config files to run the tests in jenkins and shoot the result to a sonar ?

Let's start with just a Maven. IMO it should include simple Hello World for android, integration tests for it. Of course should be possible to build project from command line (i.e. no connection with Jenkins) and run tests under emulator to obtain coverage with help of Emma. And I don't expect that this example will be tuned to push data into Sonar, because this will be next step - understand strategy to use Sonar with such projects. If build process cannot be fully automated, i.e. additional configuration of environment required (e.g. downloading and installation of Android SDK), then it should be documented.
After that we will be able to talk about next actions.
 
Good luck.

Thanks for your answer, thanks also for explaining why there doesn't seem to be so much enthousiasm in the sonar community (for now :) ).

Stéphane


2012/7/23 Evgeny Mandrikov <mandrikov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hi,

First of all - thank you for this comprehensive overview. I always was interested to try / investigate usage of Sonar for Android development, because had feeling that there is some different with typical Java development, but never really had time to do this. Your email confirms my feelings.

And indeed - I also believe that we should do an efforts to improve situation.

From your email it's pretty clear that there is several topics, which should be improved. At first let's talk about coverage:
"Emma : a mandatory code coverage analysis tool for Android" Well, I only partially agree with this statemen - indeed, this seems mandatory now, but I believe that we should try to change the world ;) and replace it by JaCoCo. For me it seems possible: Emma provides offline-bytecode-instrumentation, whereas JaCoCo online. And I suppose that  this is one of limitations, which doesn't allow to use it for Android. And here is a ticket to remove this limitation: https://sourceforge.net/apps/trac/eclemma/ticket/211

Of course this should not prevent us from making improvements in other places:
  • As I know - improvements around "integration tests coverage" are scheduled for Sonar 3.3.
  • However I didn't fully / really understand connection with integration tests coverage, so question: is it possible to use Sonar EMMA Plugin without integration tests coverage, but with fix for SONARPLUGINS-1356 ? If so, then we can quickly arrange this by making new release of plugin.
  • Regarding Lint - what prevents from a development of dedicated plugin for it? The fact that PMD, Findbugs and Checkstyle are delivered with Sonar Core, doesn't mean that other plugins can't have same level of integration into Sonar.
  • I don't know Android experts in Sonar Core team and apart from you I didn't saw other Android experts in Sonar community, but maybe I missed someone. This is just to say that preparation of examples might take a lot of time, if it will be done from scratch. And in fact this is one of the blocker factors to improve support of Android (at least for me). So I believe that it would be really valuable, if you can provide some small projects to demonstrate development for Andoird with Ant / Maven. So to allow to play with them for Sonar.
Hope this email can help to start of collaboration on subject.

On Fri, Jul 20, 2012 at 2:32 PM, Stéphane Nicolas <snicolas-07H4pXfMmVU@public.gmane.org> wrote:
Hello all,

I wrote to this list a little while ago about Android support in Sonar and, after having talked with Freddy Mallet, I realize that I should give more details in order to explain what a Sonar support for Android would mean and how Android programming differs from usual Java programming.

I will write this mail under the form of a bullet list, I do it to be as clear as possible and hope it won't be too dry/too long/too rude : 

Limitations / Differences between Android and Standard Java projects : 
  1. Android differs from Java as it doesn't include every feature of the language, most of them are but not all packages are present. I tend to believe that, for instance, this could have some impact in terms of introspection or byte code manipulation that are supported by the Android platform.
  2. The biggest difference is, for sure, the JVM of Android. Android devices use a custom JVM called Dalvik. There isn't much difference between Dalvik and a normal JVM but still, this has some impact. For instance code coverage tools that are supported by Dalvik are more limited than within a normal JVM.
Emma : a mandatory code coverage analysis tool for Android integration tests : 
  1. After talking with Freddy Mallet, I realized that Jacoco became somewhat a standard for code coverage analysis in Java projects. Jacoco being an evolution of Emma. In Android, the standard byte code instrumentation/code coverage tool is still Emma (version  2.0.5312 is shipped with release 19 of the Android SDK). Version 2.1.5320 simply doesn't work (due to Emma RT controller). 
  2. Emma can be used to run instrumented tests in Android, rather easily. Those tests will be run in a device / emulator. The way Android is built makes it very difficult to isolate software components from their contexts. By this, I mean that most tests programmers will write in the official Android way are not unit tests but integration tests. Android programmers who use the maven-android-plugin are already familiar with this idea as all Android tests are considered integration tests.
  3. Thus, in order for Sonar to fully support Android testing, it is necessary to be able to switch from Jacoco to Emma as the IT code coverage tool. Unfortunatelly, Jacoco being a standard, Sonar doesn't offer any mecanism to use a different code coverage tool for integration tests, one can change the unit tests code coverage tool, but the integration tests code coverage tool is Jacoco and it can't be changed. 
  4. With regards to this consideration, I submitted a fork to Sonar project on Github to be able to swith from Jacoco to any other code coverage tool. I am not sure the fork is very well done (indeed, my main problem is that I didn't understand the tests), but still this idea works. I compiled a custom sonar version with it and was able to use Emma as the IT code coverage engine. Simon Brandhof already answered to me (on dev <at> sonar) that this patch could not be the best way to achieve my goal, but no alternative has been proposed yet.
The Emma plugin should be revised : 
  1. If the previous patch, or any alternate solution, would be accepted, the Emma plugin for Sonar should be revised in order to support Emma being used as It code coverage, not only unit tests. I already forked the Emma plugin (although I don't know where to publish the fork). And I could achieve Emma Sonar Plugin to provide Integration test code coverage.
  2. The plugin also misses some transcoding capability to support Emma. The issue has been raised and a solution has been provided but I don't think the issue has been fully adressed yet. I had to use a custom Emma task in order to use it with my android project inside Sonar.
Unit tests in Android : a solution using Roboelectric. An undercover sonar plugin is needed.
  1. As I mentionned earlier, the architecture of the Android Platform makes it difficult to design code that is fully decoupled from the Android Platform itself. Most Android code is tied to the execution context of the platform. Thus Android tests are mostly Integration tests and not true unit tests as they can only be run on an actual device / emulator.
  2. Nevertheless, a solution to this limitation is to use Roboelectric. Roboelectric provides stubs/proxies for most Android classes. Those stubs can be run inside a normal JVM and offer a great improvement in speed. As such, roboelectric tests offer the ability to truly unit test code, by stubbing the Android context classes your code relies on.
  3. Unfortunately, Roboelectric uses strong byte code manipulation that prevent it from being used with Jacoco, Emma, or Covertura code analysis tool. Nevertheless, one code coverage tool : undercover, is compatible with roboelectric. 
  4. I submitted a fork to undercover in order to be able to use it with Sonar. The fork works well but has not been integrated to undercover yet and I really wonder if it should. A better idea would be to submit it as an official Sonar plugin but I don't where to submit such a project.
  5. An additional benefit of using Roboelectric is that it has its own test runner. The runner can be used to run Android tests inside a standard JVM and is compatible with most mocking frameworks (at least easy mock and powermockito). This testing configuration thus really provides a powerful environment for unit testing Android components.
Lint : an heuristic based quality analysis tool dedicated to Android programming.
  1. Lint is a powerfull tool provided by the Android SDK that analysis your Android code and checks for known potential bugs, bad programming practices, troubles in both Java code and XML Android files (layout optimization for instance).
  2. Lint should be fully integrated to Sonar and provide Quality Analysis reports in the same way as PMD, finbugs or checkstyle are. I believe that such an integration could be done easily. This is a very import feature as Lint is becoming an incredibly powerful coding companion for Android coders.
Both Maven-based and ant-based examples should be provided by Sonar
  1. Android offers native support for Ant as a build tool. Nevertheless, Ant scripts evolve very often (virtually every Android SDK released by Google changes the basis scripts) making it very time-consuming to maintain custom ant build scripts in Android projects. The community tackled this problem and proposes a maven android plugin that offers the best of maven world to the Android platform and tends to be more and more adopted. Inside Eclipse, maven support for Android is getting more and more mature (but still not that stable/usable, particularly inside Juno).
  2. Thus it would be a very good idea for Sonar to provide both Ant and Maven examples to use Sonar for Android projects. Both technologies are interesting and I tend to believe that there is no unification / standardization that can be foreseen in a near future.
Sorry for that long long xmas wish list. I have been working on Sonar Android support for a while on a professional basis (as research and development topic for Octo Technology) and almost every aspect of this work is summarized in this e-mail.

To conclude, I hope that this e-mail can be useful to Sonar team to understand better the challenges behind Android support and provide hints, cues and tracks to achieve it. I don't know how things work here, but I would be tremendously interested in a collective effort to make Android and Sonar more integrated, as I really appreciate both of them and I am pretty sure Sonar support would help to industrialize the Android platform.

Thanks for reading,
--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
 Développeur & Consultant Android / Java
 OCTO Technology
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
Mob : (33) 6.26.32.34.09
www.octo.com
blog.octo.com
www.usievents.com
...........................................................




--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Best regards,
Evgeny Mandrikov aka Godin <http://godin.net.ru>
http://twitter.com/_godin_



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................



--
Stéphane NICOLAS,
OCTO Technology 
Développeur & Consultant Android / Java
..........................................................
50, Avenue des Champs-Elysées
75008 Paris
+33 (0)6.26.32.34.09
www.octo.com - blog.octo.com
www.usievents.com
...........................................................

Gmane