Some day we will have a full team to run various tests (integration, load testing, application compatibility, etc.). Until that day, the simple recipes below will guide a user to make sure that different parts of the application work properly before a release.

Testing Using Real-Life(R) Data

If you are looking for documentation on how to test a release candidate against your own database, see the Release Testing Helper Module page.

Testing the API Layer

It is assumed that unit tests have been written against any new API code since the last release. All unit tests are run after every commit by our Continuous Integration server. You can see how well the tests are covering the api by looking at the unit test coverage output.

Testing the Web Application UI Layer

The "release-test" mavenmodule in trunk contains selenium-over-jbehave unit tests.

There is a continuous integration project that runs these tests weekly. TODO: create project and/or add link

See the currently defined stories in code.

Read about Selenium and JBehave. TODO: add appropriate links

How to add a story

If you would like to add a story to our suite, open a new ticket with the text of the story. It should look something like this:

GivenStories: org/openmrs/stories/go_to_admin_page.story

Given I am on Admin page
When I choose to manage encounter types
And I choose to add a new encounter type
And I mention name YOUTHINITIAL and description Outpatient Youth initial visit
When I save the encounter type
Then the new encounter type should be saved

This story goes into the release-test/src/main/resources/org/openmrs/stories folder.

Creating a runnable story

JBehave looks for a java class with the same name as the .story class. So from our example above we would need a class named CreateEncounterType.java in the org.openmrs.stories package.

public class CreateEncounterType extends Story {
	@Override
    public List<Steps> includeSteps() {
        return asList(new LoginSteps(driver), new AdminSteps(), new CreateEncounterTypeSteps(driver));
    }
}

The LoginSteps and AdminSteps are required for the parts of the story covered in our required go_to_admin_page.story.

Our CreateEncounterTypeSteps code and annotations should match our story:

@Given("I am on Admin page")
@When("I choose to manage encounter types")
@When("I choose to add a new encounter type")
@When("I mention name $name and description $description")
@When("I save the encounter type")
@Then("the new encounter type should be saved")

The text must match up exactly or the story will not be able to run.

See CreateEncounterTypeSteps.java for full code.

How to run the stories

Tests are run from the main openmrs root directory. This is because maven must launch a jetty server using the webapp project code.

Since openmrs is running in memory, you need to increase the amount of memory given to maven:

export MAVEN_OPTS="-Xmx512m -Xms256m -XX:MaxPermSize=256m"

Using the release-test.sh script

Execute this script at a command line with:

sh release_test/release-test.sh -b -v -wMyOpenmrsPassword -tGoToAdminPage

This runs the test in the background (-b) so no browser pops up (requires xvfb to be installed), gives verbose output (-v) uses "MyOpenmrsPassword" and "admin" as the credentials in openmrs, and only runs the GoToAdminPage story/steps.

This requires an openmrs database to be installed into a running mysql that can be found using the openmrs runtime properites.

If you want ALL tests run against an empty in-memory (requires that the initialization step be run, which takes 10 mins), use the -i flag.

Running directly using maven

mvn clean integration-test -DskipTests -P integration-test

or

mvn clean integration-test -DskipTests -P smoke

You must skip tests because jbehave runs these as classes, not as Junit tests.

"integration-test" means run ALL tests against an empty in-memory db (which means the intialization step must be run, which takes 10 mins).

"smoke" means run against a pre-defined mysql database that already contains openmrs code. A runtime properties file must point at that.

Possible stories that still need to be coded up

TODO: this list needs to be compared with the currently stories. Some of these might be completed already.

• void the whole encounter - make sure the obs get voided too
• unvoid that encounter - make sure that the obs get unvoided too
• Forms (InfoPath, htmlform & xforms) (module dependent)
  • Duplicate a form
  • Add a new field to a form schema
• Concepts
  • Update All Concept Words
  • Update a range of concept words
  • Add/change/delete concept
  • Add/change/delete concept map
  • Add/change/delete concept class
• Scheduler tasks (make sure they execute)
• Message service (smtp)
• Merge patient records
• Create/modify program/workflow/state
• Patient dashboard
  • Review header for name, age, ids, bmi, regimens, last encounter, etc.
  • Overview tab
    • Add/delete patient identifiers
    • Enroll patients in program/workflow/state
    • Add/delete relationships
    • Add/delete allergies
    • Add/delete problem list
  • Regimen tab
    • Add/stop regimen and drugs
  • Encounter tab
    • View an encounter on the dashboard
  • Demographics tab
    • Edit patient identifier
    • Edit address
    • Edit name
    • Edit patient attributes
    • Edit with short and long form
    • Edit deceased along with date of death
  • Graph tab
    • Add CD4 count along with obs
• Run cohort builder
• Create/edit data export
• Manage users
  • Create new user roles
  • Modify privileges for existing user roles
• Manage relationships
  • Add a new relationship
  • Change display order
• Patient attribute
  • Add new patient attribute with various formats (including org.openmrs.concept)
• Encounters
  • Create encounter location
  • Create encounter type
  • Retire/purge encounter type
• Locations
  • Change location name
  • Add/retire location
• Manage drugs
  • Add/change/retire drug
  • Check adding concept drug
• Global properties
  • Add/change global property
  • Change theme and locale 
• Modules
  • Install module from repository
  • Install module from file
  • Update module
  • Start/stop module
  • Check for upgrades
• Test that the key features of the bundled modules are not broken, see below for details for the features to test for some of the bundled modules

Features of Bundled Modules

Its possible in

*Logic

• • To be updated...
    *Form Entry
  • Create a form
  • Fill out a form and submit it. Verify the new encounter is added to the patient
  • Test the search feature in the task pane (done when selecting the provider while filling out the form)
  • Upload an XSN file
    *HTML Form Entry
  • Create a form
  • Fill out a form and submit it
  • View entered form
  • Edit previously entered form
  • Check for text boxes, dates, check boxes, radio buttons, drop down menus, missing required value
  • If a required value is missing (ie.  encounterProvider), does it reset the other selected values?
  • Has the new release slowed the load time of an htmlform?
  • Do the unit tests all run correctly?
• Reporting
  • Generate a report (need more details here)
• Reporting Compatibility (cohort builder)
  • Create a cohort of all male patients with weight > 50
  • Save that cohort as a dynamic cohort
  • Save that cohort as a static cohort
  • Load the dynamic cohort
  • Load the static cohort
  • Create a data export and run it
• Serialization.xstream
  • Nothing to do here, tested while doing reporting tests...
  • Confirm that it starts
• Xforms
  • Create form
  • Fill out a form and submit it

Features of other modules

These modules are used widely and could benefit from having stories written for them.

• HTML Widgets
  • Nothing to do here, tested while doing reporting tests...
• IDGEN
  • Do all of the configuration pages open correctly?
  • Configure a sequential number generator and use the idgen 'view' page to generate a batch of IDs.
  • NOTE:  a conversation with Mike Seaton this morning makes me think that this will be broken in 1.8
• Metadata sharing
  • Import file from another server
  • Create metadata sharing zip
  • Download and import to other server
• MRNGen
  • Is the 'create patient' identifier portlet correctly overwritten?
  • Use mrngen to successfully to autogenerate an identifier while creating a patient.
• Patient summary
  • Does module start?
  • Look at patient dashboard (overview tab) for "Patient summary" link
  • Look at the summary.
• BIRT reporting
  • Start module
  • Make certain all the global properties are correct
  • etc.
• SimpleLabEntry
  • This module extends dojo for some of its functionality so this will definitely be broken in 1.8
  • The test is to 1) configure the module, 2) create an order, 3) register a result for that order
  • And, do the 'create patient' mechanisms in the module successfully create a patient?

Other Important Links

Behavior Driven Development In OpenMRS