Friday, October 26, 2012

Git and Maven release:prepare

I don't know if it happens to me only due to my complex source control configuration (which uses SSH keys like Github), but it happens quite often to me that the release process gets stuck right when it's time to push changes.

When this happens you are left with almost no choice but to perform a release:rollback and start again, but today I've found a good solution to this problem: git reset.

So, if your release preparation process is interrupted for some reason and you want to restart it again, but when you try you get a message saying the git commit command failed, then just do the following:

git reset --soft HEAD~1

This command will remove the commit from your local clone leaving all the Maven changes intact, thus allowing you to recover the process where you left it.

Have fun!

Thursday, October 25, 2012

Jacoco code coverage tool

Today I discovered with great pleasure a great project for gathering unit, integration and automation tests code coverage: Jacoco!

I was so excited that I decided to give it a try on my template Maven project and the POM went down 150 lines with greatly improved readability!

No more tricky profiles or instrumented artifacts you need to include explicitly and solely to gather coverage: everything works like a charm!

Obviously my previous post regarding Cucumber tricks is still totally valid.

Thanks Jacoco!

Thursday, October 18, 2012

Gherkin syntax quicksheet

I found an interesting article containing a nicely presented and very well described Gherking syntax guide and I want to support it.

Wednesday, October 17, 2012

Embedding screenshots in Cucumber JVM

I was trying to improve our automated test suite and I thought that it could be useful to capture a screenshot of the browser whenever a test fails.
The current Cucumber JVM implementation highly simplifies this task, but the task is not achieved the way I thought, so this is the reason for this post.

Normally you would use a JUnit TestRule to augment all your test cases with a feature to take screenshots, but for Cucumber JVM it's much easier thankfully to the Execution Hooks:

 public class ScreenshotHook {  
     private WebDriver driver;
     public void embedScreenshot(Scenario scenario) {  
         if (scenario.isFailed()) {  
             try {  
                 byte[] screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);  
                 scenario.embed(screenshot, "image/png");  
             } catch (WebDriverException wde) {  
             } catch (ClassCastException cce) {  

As you can see I'm injecting the WebDriver instance (I'm using the Cucumber JVM Spring integration) and defining an embedScreenshot method which is annotated with cucumber.annotation.After: this is the important bit because methods annotated as such will be executed after each cucumber scenario.

For sake of completeness I want to tell you there is another hook cucumber.annotation.Before that can be used for things like logging into your application eliminating those repetitive log in steps.

UPDATE: Scenario was ScenarioResult in pre 1.0.0 versions of Cucumber JVM.

Tuesday, October 16, 2012

Agile and the Definition of Quality

If you are interested in software quality, if you want to improve your agile process output, if you like to have satisfied customers you want to read this article and, after reading it once, you might want to read it a second time: what he's saying is not obvious at all.

Wednesday, October 10, 2012

Selenium tests on Jenkins

It's not uncommon to have a CI server running tests based on Selenium and it's not uncommon to get in troubles with Linux headless (without a windows manager) servers.

Usually you get errors like:
Error: no display specified
Error: cannot open display: :0.0

The following is what I think is a solution that should be portable and does not require a lot of configuration skills.
First of all you need to install the Jenkins Xvfb Plugin, which allows you to easily start a new virtual screen per  Jenkins job. Please do not forget to properly configure this plugin in the Jenkins System Configuration screen as described in the plugin page: if you don't know the path of the Xvfb executable you just run the following command in a shell on your CI server
 which Xvfb
Usually the value you will use as Directory in which to find the Xvfb executable is /usr/bin.

Once you have this you just need to activate the Xvfb plugin for your job and that should be it, without any modification in your POMs or any other hassle: the plugin will create a new virtual screen, set the DISPLAY environment variable and execute your build.

If you are using the FirefoxDriver, as I do, and you still have issues, but you can see in your output lines like the following ones, then your problems are not any longer related with being on a headless server and my suggestion is to ensure you are using the latest version of the selenium drivers:
Xlib:  extension "RANDR" missing on display ":32".
failed to create drawable
*** LOG addons.xpi: startup
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: /tmp/.../webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found

Please note the initial two lines are perfectly fine and not related to your issues!
I struggled with an error which actually went away right after I updated my selenium-java dependency to the current latest!

Thursday, August 30, 2012

A few Cucumber tricks

I've been a fan of TDD since I discovered it a few years ago. Thankfully to my friend and colleague Augusto a few months ago I've discovered the beauty of BDD and ATDD: I got so fascinated by this practice I started an Open Source project for an Eclipse plugin featuring a rich editor for Cucumber feature files. It's called Natural and I think it's pretty cool if you want to check it out.

After having used Cucumber for a while and many, many mistakes I now have got some experience I wish to share with you.

One of the key features of feature files is that they represent a live project documentation, something developers write and maintain and everyone can understand. We believe so much in this concept that we wanted to give our product owner and business partners a nice view on those files, that's why we used Relish to publish them in a pretty colored format. At the same time we had to face some problems while practically use these files for automation testing.

After a few development cycles we were so excited by Cucumber and ATDD that our automation test suite counted hundreds of tests: it was consuming too much time to be completely executed on each commit by our CI system.
My suggestion is to track in your feature files the development cycle each test has been developed: this can be easily achieved by using a tag, something like @sprint-7.
This way you can instruct your CI system to execute the current sprint automation on each commit and schedule a full test nighttime (and launch time possibly).

Another issue was we were developing automation test before the functionality was implemented, so following TDD principles. By doing this our CI was reporting a build failure until the functionality was completely implemented. This is not a problem as it is expected to have failing tests during development, but the continuous failure messages received by the team members brought us to the point we were starting to ignore those messages, vanquishing the purpose of the automation suite and the CI system itself. The solution I suggest is to temporarily annotate the tests that are expected to fail with @future, instructing your CI to skip any test annotated as such: the developers can still execute those tests, but the CI will just ignore them. Once a feature is completed it will be developer's responsibility to remove such annotation.

With hundreds of tests we had problems in having the tests properly organized so here comes the folder structure and naming convention. Please consider those files not as test, but as your application documentation: as such you want to organize then in a manner they are easily accessible by a business person. My suggestion is to organize the tests in files describing different features, grouping the features into folders representing functional areas. Considering company search and report search capabilities I can imagine a search.feature and an advanced-search.feature, both inside a company folder and the same file names in a report folder.

Defect and user stories tracking, if desirable, can be once again achieved by using tags like @US1234 and @DE9876, but I would rather avoid cluttering the feature files with such information as they will tend to distract a business person from the real value of those files: application documentation.

Automation test coverage is a report we found business users find quite valuable, but it was tricky to achieve considering our application was a web one. Nevertheless we managed to have it running on our CI thankfully to Maven and its amazing set of plugins.

I've prepared a template Maven project for those reading this post to use as a starting point and guideline if you like the solutions we adopted: it will definitely help me in the future to avoid redoing all the steps from scratch! Please, read the README file before asking for clarifications :-)

Thursday, February 23, 2012

Maven POM version management

A friend and colleague pointed me to a great Maven plugin I wasn't aware of and that is probably going to save me some headaches: the Codehaus hosted versions-maven-plugin brings to Maven users some very nice functionality!

I'm not going to copy here the full documentation, but just to make you wishing more, here is the goals list:

  • versions:compare-dependencies compares the dependency versions of the current project to the dependency management section of a remote project.
  • versions:display-dependency-updates scans a project's dependencies and produces a report of those dependencies which have newer versions available.
  • versions:display-plugin-updates scans a project's plugins and produces a report of those plugins which have newer versions available.
  • versions:display-property-updates scans a projectand produces a report of those properties which are used to control artifact versions and which properies have newer versions available.
  • versions:update-parent updates the parent section of a project so that it references the newest available version. For example, if you use a corporate root POM, this goal can be helpful if you need to ensure you are using the latest version of the corporate root POM.
  • versions:update-properties updates properties defined in a project so that they correspond to the latest available version of specific dependencies. This can be useful if a suite of dependencies must all be locked to one version.
  • versions:update-child-modules updates the parent section of the child modules of a project so the version matches the version of the current project. For example, if you have an aggregator pom that is also the parent for the projects that it aggregates and the children and parent versions get out of sync, this mojo can help fix the versions of the child modules. (Note you may need to invoke Maven with the -N option in order to run this goal if your project is broken so badly that it cannot build because of the version mis-match).
  • versions:lock-snapshots searches the pom for all -SNAPSHOT versions and replaces them with the current timestamp version of that -SNAPSHOT, e.g. -20090327.172306-4
  • versions:unlock-snapshots searches the pom for all timestamp locked snapshot versions and replaces them with -SNAPSHOT.
  • versions:resolve-ranges finds dependencies using version ranges and resolves the range to the specific version being used.
  • versions:set can be used to set the project version from the command line.
  • versions:use-releases searches the pom for all -SNAPSHOT versions which have been released and replaces them with the corresponding release version.
  • versions:use-next-releases searches the pom for all non-SNAPSHOT versions which have been a newer release and replaces them with the next release version.
  • versions:use-latest-releases searches the pom for all non-SNAPSHOT versions which have been a newer release and replaces them with the latest release version.
  • versions:use-next-snapshots searches the pom for all non-SNAPSHOT versions which have been a newer -SNAPSHOT version and replaces them with the next -SNAPSHOT version.
  • versions:use-latest-snapshots searches the pom for all non-SNAPSHOT versions which have been a newer -SNAPSHOT version and replaces them with the latest -SNAPSHOT version.
  • versions:use-next-versions searches the pom for all versions which have been a newer version and replaces them with the next version.
  • versions:use-latest-versions searches the pom for all versions which have been a newer version and replaces them with the latest version.
  • versions:commit removes the pom.xml.versionsBackup files. Forms one half of the built-in "Poor Man's SCM".
  • versions:revert restores the pom.xml files from the pom.xml.versionsBackup files. Forms one half of the built-in "Poor Man's SCM".