Interested is participating in the Google Summer of Code program? Please see the GSOC wiki page.

Wiki Spaces

Documentation
Projects
Resources

Get Help from Others

Q&A: Ask OpenMRS »
Discussion: OpenMRS Talk »
Real-Time: IRC Chat

Projects

Skip to end of metadata
Go to start of metadata

Tech Lead

 

BA Lead

 

Abstract

As an open-source community we are driven and sustained by the contributions that people make every day. We always encourage people to be more involved, and exhibit various behaviors, and we have ideas about what makes our community successful. But we do a really bad job of measuring our overall community health, and the behaviors of individual community members, and thus we don't really know whether things are going well or poorly, except anecdotally.

The goal of this project is to gather the bits of community activity that are taking place every day on the web everyday, consolidate these into a single database for analytics, and generate KPIs that can help drive our organizational behavior.

Project Champions

Objectives

Business Analysis:

  • Define 5 KPIs that we would like to use to measure OpenMRS's community health
  • Define a list of activities that contributors should get "points" for carrying out
  • Mock up a public "OpenMRS Community Status" dashboard 

Technical:

  • Create a single database where we can store activity
  • Connectors to pull activity data from GitHub, JIRA, IRC, etc. 
  • Database to store our calculated KPI scores, and contributor scores 
  • Create the "OpenMRS Community Status" Dashboard

Brainstormed Ideas

OpenMRS Community KPI

  • total monthly number of code commits (year on year)

  • # of people scoring "good" or above on an individual contribution metric
    • review the specific people who were "good" for the prior two months but not this month, for potential individual followup
  • % of dev activity that is happening in priority areas (priority = identified by the Implementation team as priorities? or is part of an active sprint?)
  • % of posts to "Ask OpenMRS" that receive a reply within 3 days; % of posts to "Ask OpenMRS" that receive an accepted answer within 7 days
  • # of tweets with #openmrs in the last week
  • # of people (or people-days) who participated in an IRC scrum in the last week

 

Individual Contributor Scoring

  • said something on IRC between !scrumon and !scrumoff (+10)

  • said something on IRC (+1)

  • Any Talk post (+1)

  • Reply to the Ask OpenMRS category on Talk (+5)

  • Reply to Ask OpenMRS with a link to the wiki (+10)

  • commented on an open github PR (+1)

  • merged someone else's PR (+10)

  • had your PR merged (+10)

  • contributed a translation on transifex (+1)

  • github commit to any /openmrs repo (+1)

  • github commit to any repo with an active sprint (+5)

  • created a new wiki page (+5)

  • edited a wiki page (+5)

  • openmrs twitter interactions (+0.5 per tweet)

  • commented on a wiki page (+0.5)

  • created a site on openmrs atlas (+5)

  • added unit tests (+2 per commit)

  • pull request broke the build on CI ( -2)

  • pull request merged with no CI failure (+1)

  • creates an openmrs id (+5)

  • forks an OpenMRS repository (+5)

  • attends an OpenMRS Design call (+5)

  • writes notes for an OpenMRS meeting in notes.openmrs.org (+2)

  • # of reported JIRA issues (claimed and closed issues are probably better reflected in created and merged pull requests)

  • # of conducted reviews (# of all comments in github). At the moment we don't have a way to accredit reviewers, thus only those who commit or merge get the credit. It would be nice to track reviewers in JIRA issues in a dedicated field and a state in workflow

  • # of comments in JIRA

 

Tech Notes

These days it is really easy to consolidate and analyze data, and there are great modern technologies for doing this. I did a quick spike of this idea using ElasticSearch, NodeJS, and AngularJS, and we'll likely use similar technologies for this project (unless we find an off-the-shelf suitable solution to this problem).

Broadly, we want:

  • a NoSQL DB to store our "community activity", used in an append-only style
  • a few feed readers +/- an IRC bot to harvest data from github, JIRA, Talk, etc.
  • a server-side app with scheduled processes to calculate and store our metrics (should use js or groovy for scripting the calculations)
  • a small html+js app that we run at some url like om.rs/communitystatus which publicly displays our KPI trends and highlights top contributors (recent, and all-time)

See my spike at: https://github.com/djazayeri/openmrs-contrib-analyzecontributions

It's possible there is an off-the-shelf solution for what we want to do. We should look into:

 

  • No labels

4 Comments

  1. Great Idea.It seems interesting.

  2. I was talking to Saptarshi Purkayastha about this yesterday, and we think there might be a way to leverage his starting work on Discourse badge plugin for GitHub activity. Many of the things we want to measure are part of OpenMRS Developer Stages and so we could create interim/sub-stage badges as a way to look for certain types of behavior – not only on Talk but also in GitHub or maybe even elsewhere. Need Ruby skills for those Discourse plugins, though.

    1. Cool thought! One thing is that the Developer Stages are an achievement you reach once, whereas I think we're interested in someone's current/ongoing level of activity.

      There are lots of ways to approach this, and I definitely have a preference for JavaScript/Java, as these will let us leverage more of the OpenMRS community. Still, I'm sure there's a way to mix and match as well, e.g. as long as we get the behaviors recorded on a database.

  3. Darius Jazayeri, I hope it's okay that I moved this project from "active" to "unassigned". I added a link to GrimoireLab documentation we received today and thought we might be able to morph this project into something where someone could volunteer to work on implementing some of the Bitergia tools for the community.