Jira Issue Curation

Tags: #<Tag:0x00007f0f1a0cd958>

Hi everyone!

As you may or may not have realised, our Jira instance has quite a lot of cruft in it, so we (the Technical Action Committee) have been trying to think through some ways we might improve things. I’ve put together a document with some thoughts on some steps we might take, which you can find here. Please feel free to comment here or on the document with any insights, criticisms or other feedback you might have!

Thanks!

@burke @grace

7 Likes

Thanks @ibacher for putting this together. Very clear.
I’d love to see us start trying out these ideas. It should help make our sprint and project planning workflows a lot more clear. Right now it’s really hard to see what we should be working on next, which tickets are pressing vs. just noise, etc.

The GitHub Issues idea is an interesting suggestion. What do others think about using GitHub Issues instead of Jira? I am very familiar with Jira, and it does have a lot more tools to support planning and reports, but I’m happy to learn new tools too. My inclination is for us to try improving Jira together first, but I’d like to hear from others.

3 Likes

Migrating from Jira would first of all require accessing the features we lose and gain in the move. Then the actual migration, and its associated activities like learning the new tool usage, documentation changes, etc. Looking at our currently limited resources, i would not divert them into a process which does not have significant benefits to be worth the effort.

6 Likes

I do think we should give strong consideration to moving to JIRA cloud. JIRA has gotten much, much better since the last time our server was updated.

We’re not really that far behind on things software-wise (we’re on 8.4; latest is 8.9). Are there specific features we are missing?

1 Like

Oh wow, maybe there was an update since the last time I noticed. Or maybe I just notice the presence or absence of Markdown way more than anything else. Jira Cloud has incomplete Markdown support; Jira Server (at least the version we use) has none. Don’t worry about it, anyway.

Actually, real Markdown support would be a strong motivator for me. I constantly get bitten by switching between GitHub Markdown and Jira’s custom mark-up formatting.

2 Likes

The latest version of Jira does feel quite different when you first start using it. It took me a long time to decide it was better. There are some UI changes that at first feel totally un-intuitive to figure out, and then over time the workflows feel easier than before.

How much of a lift is it to do this transition from our hosted Jira to the latest cloud version? If it’s more than a day of work for 1 person, I don’t think it’s worth it right now just for the UI changes and Markdown. But likely after we’ve tried out changes to our Jira workflows :slight_smile: One thing at a time.

I brought up the idea of considering GitHub as we re-evaluate how we curate issues primarily for two reasons:

  1. If we felt the pros outweighed the cons, it would be better to adopt GitHub issues before putting a lot of effort into improving JIRA rather than after :slightly_smiling_face:
  2. Organizations and OpenMRS Squads are already using GitHub Issues in certain cases, which means we’re heading toward a world with both JIRA & GitHub issues in the community

Daniel makes a great point that we have limited resources, so need to be thoughtful. That said, I don’t think this automatically means JIRA should win because it’s the path of least resistance. The status quo includes ongoing infrastructure resources and persists an OpenMRS-specific approach with additional hurdles (learning OpenMRS issue workflows, barriers to creating a new project, etc.).

Personally, I like to think of this from the developer’s perspective (both seasoned & new to the community). We may end up sticking with JIRA, but if we’re going to consider migrating to GitHub issues, this would be a good time to do it. To that end, here’s a quick brain dump of pros & cons that come to mind…

JIRA GitHub Issues
- No or relatively easy migration (e.g., to Hosted JIRA) - Migrating tickets would take more effort
- Independence from GitHub (where code is being written), can be confusing or require more effort/training - Intuitive as part of GitHub environment
- OpenMRS doing things the “OpenMRS” way - Aligns the majority of other open source projects
- Workflows can be customized, but also confusing/frustrating - Simple with tools like ZenHub
- Linking commit comments to tickets requires addons - Linkage between commits & tickets is automatic
- Can run JQL against all “OpenMRS” tickets - Presumably could do something similar with issues under OpenMRS org
- Requires infrastructure support (could be mitigated by moving to Hosted JIRA) - Provided by GitHub
- Works with OpenMRS ID - Requires a GitHub account (which may not be easily to correlate with an OpenMRS ID)
- Need to go through help desk bottleneck/delay to create a JIRA project for a new repo - Issue tracking automatically included with any repo
- Easier to apply conventions (labels, workflows) across all/many OpenMRS projects - Labels, workflows, etc. are repo-specific, so trying to adopt a standardized approach across all repos is unlikely to happen or be sustainable
- No or limited markdown support. Seriously, in 2020?!? - Markdown **ftw**!
4 Likes

Thanks for the comparison Burke! I was particularly curious about what it will be like to manage cycles without some of the dashboarding, reporting, and other sprint-based displays that Jira offers/requires, but it looks like anything fancier we really wanted we could support fairly easily with ZenHub browser extensions.

The idea of wrapping up our old backlog is very appealing, especially right now as we try to build up more regular Sprint Cycles through our squads. It’s hard to plan effectively with so much noise in Jira - but Jira does require a lot of care, which often doesn’t reflect the “real work” going on in GitHub. E.g. there’s quite a difference between some of our Jira metrics right now vs our GitHub commits.

I’m happy to try this out :slight_smile: Agreed it should be soon. What do others think about GitHub Issues? We could just try it with 1-2 squads and see how they like it?

FWIW - If we choose to stay users of Jira - Here’s a quick overview of what we did in a team where our Jira instance was bursting with old projects and noise from old tickets, and what then happened. I think this is relevant here, though changing the workflows and Issue Type options would likely cause unhelpful confusion in our case at OMRS right now. We also heavily used dashboards as a home-page for each TPM to regularly check on the “vital signs” of their projects (which in this case could be the squads in our case).

tl;dr - We could start trying GitHub issues with 1 squad and just try it. If we stick with Jira it will cost a lot of task-management time anyway, because of the nature of the tool, but it is do-able to de-clutter the backlog.

I’d love to hear more about this. I’m actually quite in favour of greatly reducing our number of issue types and workflows. I think the number of issue types and workflows makes the whole experience at bit more overwhelming for new users and, frankly, I’m mostly just doing my best guess-work at what goes where.

I think the mitigation here is that we’re basically requiring a GitHub account for anyone doing dev work and it’s worth noting that access to our Atlassian stack is not automatic just because you have an OpenMRS ID, i.e., you need to specifically request access to Jira, Confluence, and Bamboo (and, if memory serves, accounts aren’t necessarily linked across those three—I think there’s actually a separate account for Bamboo).

Linking from GitHub -> Jira actually doesn’t require any add-ons, though it does require some manual intervention to use GitHub’s Autolinked references feature. I have been adding some of these to some of our GitHub repositories. For instance, I’ve configured openmrs-core to autolink TRUNK-XXXX to the corresponding Jira issue, as can be seen in the history view.

Populating that information into Jira, however, does require addons. If we stick with Jira, I think this is something we should investigate, as we could alleviate some need for double-entry. This is also a point at which our Jira workflows could be simplified, e.g., in the standard OpenMRS workflow, we have steps for:

  • CODE REVIEW (PRE-COMMIT)
  • CODE REVIEW (INITIAL)
  • CODE REVIEW (POST-COMMIT)

Personally, since I do all my code reviews on GitHub, I rarely use any of these intermediate steps.

However easy Atlassian might’ve made the migration from Jira Server -> Jira Cloud, I’m guessing all the things around that migration are harder for instance, can we continue to proxy tickets.openmrs.org and issues.openmrs.org to our Jira Cloud instance so that we don’t break a lot of links.

Thanks Ian, FWIW you can see how we changed the workflows and Issue Types in the short PPT I linked to above. The tl;dr is it is much simpler.

Side note, TBH at the time of that big clean up, I had a bit of a vendetta against sub-tasks even though they get grouped nicely under stories/tasks, because they don’t get counted in your Jira metrics which meant our reports, sprint velocity, and dashboards were not accurately showing our PMs what work was actually happening and where the project bottlenecks really were. So we used Epics & Tasks more heavily. More importantly than “what types of issues are right to use”, I think this example is a good demonstration of how easy it is to have to micro-manage details in Jira that don’t actually result in real work being done, or in having a great grasp on what’s actually being released and committed. That happens mostly outside of Jira. So the simplicity of GitHub Issues is appealing to me for this reason. But I can go either way.

I don’t have much intelligent/informed to say about Jira linking, except that I worry that with so many projects and moving parts, we will inevitably end up with gaps in linkages and automation with GitHub. This is most costly in PM focus time as they will end up spending ++ time trying to figure out what tasks are done and just not updated (which is time they could be spending on actual product work and feature research). Then they have to bug devs to update Jira which isn’t fun or an ideal use of time for anyone. And then there’s the other cost of the occasional thing that gets marked “done” in the ticket system but didn’t actually make it off the local. Doesn’t happen too often but it sure is memorable when it does!

Not trying to sway us either way, these are just thoughts - though I’m increasingly thinking it would be neat to try out GitHub issues with 1-2 squads and see how it compares for us.

See, when I saw that in the PPT, I thought: “That’s great! We should do something like that!” That said, I realise it’s an investment of time and community buy-in to make something like that happen here.

Are there any squads or implementations already using GitHub issues who could give us a sneak peak?

1 Like

Here are some of the things I would like to know from a pilot:

  • Can we successfully handle both issues being added to the issue queue while maintaining a reasonable view of issue priority?
  • What is the best practice for ensuring that issues have been properly evaluated before someone starts working on them?
  • How do we deal with issues which end up requiring touching two different code repositories?
  • What’s the best practice for dealing with issues reported against the wrong repository?
1 Like

It sounds like it’s been a combination of community members (not using Jira) submitting issues into GitHub (and those being automatically closed by a script set up to re-direct people to Jira), and possibly some historic use of GitHub Issues by the MFE and OCL squads. We have now found that of all our 255 OpenMRS repos, 17 have issues enabled, and of all these, there are only 2 issues open, both from just 1 person.

In the technical action committee meeting today we decided pursuing a small pilot with a squad would be a good idea. I volunteer as tribute to help facilitate this. :raised_hand:

Piloting GitHub Issues

  • What do we want to know? 3 high-level things: Was the piloting team satisfied enough with the experience that they’d recommend it to others? Did it help us get work done? Was there anything we really missed compared to Jira? (@ibacher you raise some great specific constraints; maybe I could put together a sample repo project that we could experiment with to see if we can’t set up boards and labels to address most of these; what do you think of that?)
  • Which squad will pilot? Looking for volunteers :joy: I commit to helping that team learn how to use the issues and addressing problems that come up.
  • For how long? This is of course up to the squad; I think 2 cycles would be ideal, because we’re kicking off a number of cycles and squads right now, and it would be unideal to get comfortable in Jira only to have to change the tooling all over again, so the sooner the better IMHO.

@dkayiwa @ibacher @burke any thoughts?

I definitely think exploring a move to Github Issues is worthwhile, just like I think exploring a move from Bamboo to Github Actions is worthwhile in many cases.

One comment around the backlog - I’m reluctant to treat backlog as cruft just because it hasn’t been updated in a long time. In many cases, JIRA tickets represent a vision for what functionality would ideally exist for a given area, given prioritization and resources, and is there in lieu of a wiki page or other functional requirements or roadmap documents. It may well be important to identify these high level (or detailed) requirements and document them somewhere differently, but I’d worry about a process that would discard them all as historical noise.

Mike

2 Likes

So I feel like I should clarify that it’s never been my intention to treat the backlog as cruft and just say “close it and we’re done with it”. The problem is that there’s so much noise in the backlog that I think that whatever is of value there is already effectively treated as cruft. We probably do need some process to review the back log.

Maybe it would be better to start simply by closing out tickets for projects which we’ve already nominated to be archived and see where that leaves us? A number of the longest open issues are reported against things that are no longer supported, e.g., the BIRT module, the Eclipse tools, the OpenMRS Concept Collaborative, etc.).

2 Likes

@mseaton agreed with your points above. To Ian’s point that the noise is causing the backlog to be effectively treated as cruft, and needing process: whether we stay with Jira or move to GitHub Issues, we will still need to have a once-over before closing things that meet given age criteria. @dkayiwa how have we handled this in the past? I have previously tried too approaches; 1) I can go through the backlog myself and try to i.d. things that seem important, and tag people in comments with questions, however this is error prone just because of my knowledge and time-consuming to get the right feedback promptly. 2) We could have a small group session with a couple technical leads and we can run through a number of projects together, so we can close or retain issues that stand out to you and others.

Would people be up for trying Option 2 together?

Ian I think closing tickets for projects already nominated to be archived completely makes sense. @mseaton are you comfortable with this as a start?

2 Likes

First I’d want to sanity check that list of projects :slight_smile: I know Birt keeps coming up, but I’m pretty sure this is still in use in our Lesotho system, or was very recently. That doesn’t mean it will ever get maintained or worked on, just pointing it out.

I’m also not really sure how closing all of these tickets that are already in isolated projects makes any difference. If there is already a JIRA project that is just for the Birt module, then is it really helping by closing all of the tickets in this project? We could just as easily put that project into a different permission group, so only certain users would see it. Or maintain better dashboards of projects that are active, and have Birt and others just available for those who go digging.

Effective use of statuses and filters might make an equally or bigger difference. If the problem we are trying to solve is to make it easier to identify the tickets that are appropriate to work on and those that are not, assigning statuses of “Backlog Review Needed” or something, and/or filtering and sorting based on status, ticket activity, watchers, votes, other other criteria should lead to (potentially more accurate) lists of tickets.

Have we identified the projects with the biggest backlogs? I assume TRUNK/CORE is the main culprit here?

To be clear, I’m not saying we shouldn’t go through and close lots of tickets ruthlessly. I just don’t think we should start doing it blindly.