Transitioning away from Jira ? Lessons learnt

Oct. 2, 2018

Over the past year, we’ve been moving most of our projects away from Jira, in favor of GitHub issues. This blog post will go over the reasons why and lessons we learnt along the way.

It all started with an opportunity. As we were launching an entirely new project, Kids-First, our teams started discussions around the project management tool to be used during implementation. Some of our objectives, in looking at new approaches, were to see how to reinforce integration between stories and actual software code, make developers’ life easier and provide management with adequate planning and reporting features.

What’s wrong with Jira ?

Let’s get this straight from the beginning, there is nothing inherently wrong with Jira, it is a great platform (I said it!) with a wide feature-set and an extensive ecosystem of third party tools.

So why be in favor of transitioning away from Jira?

Jira, protect me from myself

Workflows can get complicated! (and yes, I'm the one to blame on this one)

In my opinion, one of the the biggest challenges with Jira is also one of its strength: Jira offers a lot of possibilities for customization. Trust me, it’s very easy to fall in the trap of wanting to configure all the nitty gritty details of one team’s operation. Want to prevent an issue from moving to the next stage if testing hasn’t been documented? Sure, no problem. Require 2 reviewers before a ticket can be completed? Of course. And the list goes on.

The challenge with a complex workflow is that it easily gets in the way of the development team, in particular around exceptions during burst period. For example, on one of our projects we required documentation instructions to be written for stories. It makes sense, right ? Now if we look back at the 6,000 issues we have on this particular project, only 175 were populated with actual documentation instructions, most of which were just links to pull requests. Not the most successful workflow rule!

GitHub on the other hand, simply doesn’t allow for advanced workflow customizations. Issues can move freely between statuses, with no possibility to add validations steps at the issue level. This gives much more freedom to the team but also helps entrust all team members. It may take a few reminders at the beginning (please don’t forget to estimate unplanned urgent issues), but it’s definitely worthwhile.

Jira for Open-Source projects

When developing an open source project, it might be nice to give full visibility to the public on features being worked on, their status, and allow external people to comment on existing issues or create their own tickets. Not only opening the code, but also the project management behind it.

Although Jira can be configured for Public Access, over the past years, GitHub progressively became the de-facto platform (with GitLab and BitBucket on a smaller scale) for open source projects. Being the platform of choice for most developers, selecting GitHub for a project is one less hurdle to receiving contributions from the general public.

Plus, with GitHub, you get access to GitHub extensive infrastructure & API (more on that in another blog post) and third party tools.

Are GitHub issues the ideal solution ?

GitHub provides features for project management out of the box but we quickly realized those would not be sufficient for our needs.

Cross-organizations project boards

Although you can use a GitHub project board across multiple repositories of the same organization, GitHub projects does not allow for cross-organization boards, which is a definite no-go in our context.

We have one GitHub organization for Overture and one organization per project. Project-specific implementation go into the project organization (for example: ICGC, Kids-First, HCMI), reusable components go into the Overture organization. Each scrum team (one per project) uses its own Agile scrum board, and the lack of cross-organizational project boards would definitely be a blocker (more on that later).

Some of Overture’s issues on GitHub projects

Story Points

GitHub doesn’t have a dedicated field to store Story Points, neither does it provide metrics around those (velocity & burndown in particular). A workaround for points recording is to use Labels but there are no built-in tools for charting.

Can you spot points?

No Issues Hierarchy

GitHub doesn’t make the traditional Epic -> Story -> SubTask hierarchy obvious.

The recommended approach when working with GitHub issues is as follow:

  • Epic: Regular issue attached to an “Epic” label, Stories linked in the issue description.
  • Story: Regular issue
  • SubTask: List in the description of the parent Story, with checkboxes ( - [ ] ).

Labels Management

Labels are managed at the repository level on GitHub, and although Labels can easily be created at any point in time in the interface, there is no provided feature to ensure their consistency across repositories and organizations. So it’s easy to end up with different colors for the same label, labels with a similar but different name, …

Some labels inconsistencies across our repositories

Having the option for defining labels at the Organization level would be a nice addition.

Sprints

Milestones are the recommended approach to manage sprints with GitHub, with one small caveat though, there is no start-date for milestones (only end date). It’s not a huge deal, but a slight annoyance as you cannot precisely determine the beginning of a sprint to get its burndown.

Milestone creation in GitHub
In GitHub, an issue link burried in less critical events

Although GitHub makes it very easy to link issues together, there is no summarized view (as in Jira). Users either have to scroll through a potentially long list of issue’s events or manually add those links into the issue’s description (but in that case, not showing its state).

Jira's summarized view of issue links

I miss JQL though

Jira has its own internal query language called JQL, sadly there is nothing as powerful available in GitHub ecosystem.

GitHub's issues search

Attachments

Oh! and one last caveat with GitHub issues, uploading images works great, but that’s the only type of file that can be attached to an issue. I know some people who really miss their CSV/TSV attachments.

GitHub’s very extensive API

But GitHub played it very smart by providing 2 very extensive APIs (v3 using REST , v4 using GraphQL), their associated libraries and a live instance of GraphiQL making it extremely simple to extend GitHub functionality to cover particular use cases.

How many issues in Arranger ?

GitHub provides a strong foundation, and opens the door to third party vendors to extend their core functionalities, encouraging an ecosystem of applications to grow alongside (we’ll see how this plays with Microsoft joining the game).

Third party apps to the rescue

This is a good point to start introducing third party solutions (such as Zenhub, Waffle.io), which, by wrapping metadata around GitHub issues, extend functionalities towards Agile and project management.

Running sprint in Kids-First' Zenhub board

Over the past few months we’ve been using the two platforms mentioned above. Zenhub is extensive but sometime a bit heavy on the interface, Waffle.io has a lighter interface, but is not as extensive feature-wise. I’m not going to present those solution in details, just cover some challenges using them, especially thinking longer term.

Running sprint in Overture's Waffle board

So while using those, things started to get a bit challenging as I wanted to dive a bit more into velocity metrics and forecasting (see my related post on Jira velocity).

Accessing Agile metadata

Both ZenHub and Waffle.io have a documented API, but with limitations (some of them being pretty annoying).

Let’s assume one wants to build an app, that will go through all the issues to collect their individual Story Points in order to build metrics, things get slightly challenging:

  • ZenHub has an API rate limit of 100 requests per minute to the API with no bulk means of collecting points, it has to be done for each individual issue. Kids-First + Overture account for 2,000+ issues over less than a year. An initial load will be pretty lengthy.
  • Waffle.io is very unclear about their API usage, and the API has been labelled “private alpha test”. For the first few months I was able to bulk GET the data since our projects were open and waffle didn’t have restrictions on those (this was closed down in September). But the major challenge is individual archived issues cannot be accessed after 30 days, so all individual historical Story Points are basically gone, sigh …

Furthermore, someone from the general public navigating our repositories on GitHub would not be able to see any of this, sometime valuable, metadata.

So is this dependency to third party platforms something we have to live with, or is there a better option?

Solution?

Understandably, GitHub cannot cover for all possible business needs with their API, and it’s of course preferable to have a strong and stable API instead something unstable with a ton of options.

But does this call for a schema-less user-defined JSON info field directly attached to an issue node in GitHub database? That would be a great improvement allowing third party apps to store most of their metadata there.

Conclusion

At the moment there isn’t an ideal tool. We are ok (I’m not going to say happy, because of the issues above) with a mix between third party apps and native GitHub, but I’m looking at ways to bring more into core GitHub issues (through labels for example) to limit the above-mentioned long-run dependency.

From a project manager point of view, I like the approach and simplicity of it (less is more). In a way, it’s simplicity brings it much closer to a traditional “post-it” Agile board. Most of the key metrics are there and it’s ease of access for our users makes it a great solution for the time being.

We constantly adjust our Agile methodology & tools to best fit our needs, so with no doubts thing will change in the future, but for the time being this is an efficient and workable setup.

So GitHub or Jira for my project?

Both solutions are perfectly fine and mostly depends on your use case. If you need strict control in your workflow, Jira is probably your best bet. If you want flexibility while giving your team a sense of ownership towards their workflow, then GitHub issues is a great option.

Jira ?
or GitHub ?

I don’t have an answer for you, but hopefully this blog post would have helped identifying what might work best for you.

PS: Thanks to Rosi and Solomon for their input while writing this blog post.

Francois Gerthoffert, Project Manager
Passionate project manager, diehard curious and solutions finder. Enjoy learning and experiencing new things, from rollerblading to Niagara falls to making his own furniture. Did I hear someone talking about cheese ?