2016-07-21 Leadership Call: Infrastructure issues

Community
,
1

I’d like to respond to some of these points in the notes regarding the infra team, my comments are the second level of the bulleted list.

  • Jan and Pascal were talking about bringing more people on to the OpenMRS Infrastructure Team.
  • Yes.
  • Pascal is doing onboarding with Robby next week and thought it would be helpful to Maurya is doing work on infrustructure and would like to have him present on next weeks call
  • Thanks for keeping me in the loop…with regards to Maurya…
  • Paul was seeing some people asking to take part in infrastructure and being told they are too new
  • Experience matters. That is unless you want servers compromised and your services to go down. So yes, experience matters, so if somebody is too new, and we know nothing, i’m not handing them access to production systems. It’s basic security here.
  • Think we need to have a call with the infrastrucutre team so we can all get on the same page
  • No calls will happen. Keep it public. We’re in different time-zones. it is becoming crystal clear that you wanna make decisions without us. I don’t wanna join a weekly phone call, I hate being on the phone. While I appreciate what @burke is doing, it’s not helping us by keeping us in the dark and making decisions on our behalf. I apologize for the tone here, but this is something that I have asked to be kept in the loop about and have not been and am quite frankly getting annoyed.
  • 30-40 servers for OpenMRS hosted at IU for free - CAITS group here at IU maintain this
  • Great, again why I was I not kept in the loop – this all hidden in etherpad. None of us are kept in the loop, this is bad.
  • Leadership is looking to grow the Infrastructure Team
  • I’d love that – but don’t get desperate and take anyone who offers…we need experience. We will deny people who are new – they need to show what they have to offer…and establish trust.
  • Next Steps - Burke is going to work to schedule a call about dev ops support
  • Great, Could you include us?

Please stop doing things on the infra team’s behalf without including us.

While I appreciate @maurya coming onboard, nobody spoke to ANY of us – @maany, myself, anyone. He’s documenting things blindly. What is he documenting? He’s not even in the Infra chat, nor does he know much about the infra team structure or how we do things…so I’m just confused here. UberConference is not open, nor is it public.

2

Experience does matter, but I think the fact that we have a problem bringing new people into the process is a sign that our process needs work and our documentation needs work. With the concern of vetting people for experience, that should be part of the onboarding process - and then giving them tasks based on that experience - not excluding them from the work or team.

Unfortunately, Talk and IRC and Telegram aren’t always the best means for communication. When working with a larger group, it just sometimes is best to have real-time conversation once in awhile to fully understand where improvements need to be made and to be in agreement on direction. Uberconference is recorded and available publicly. If we announce when the call will happen, that is public as well and anyone interested can join. It’s not up to a single person to decide how members in the community communicate. If a group of people want to have a real time conversation, and one person refuses to join that, then that’s okay - the group can still proceed with a call and share their conversation in another format as a summary or talk thread or any other way for others to catch up.

Many of us have to use a variety of methods because of situations we are in and timing constraints. We must accept that some of our conversations happen organically because we work together over a variety of projects and talk through multiple things together when the opportunity arises. It’s not intended to exclude anyone or make decisions without anyone else. As long as those conversations and decisions come back through the community, then it’s public. There are many Talk threads, uberconference discussions, etc that are happening regarding infrastructure that people can take part in - no one is being kept in the dark. We all have the same goals here.

Etherpad is a public documentation of the discussions, same as Talk. Just a different format that can be used real-time during calls so everyone can contribute to the documentation of the meeting.

The vetting of people should not be “deny people who are new” but rather “let’s discover what they have to offer in skills and judgment and get them working at the right level based on that.”

@maurya is helping better understand about what folks want out of the helpdesk application. He’s just doing some initial research and will be talking about it next week. There is plenty of time for comment and input and feedback from everyone. It’s great work and I expect you to be supportive of him doing it because it will really help the infrastructure team and those using the helpdesk.

1 Like
3

Thanks @r0bby for keeping us honest. The truth is, we all want to help out and we don’t have time to monitor Talk and type everything into it. I would prefer not to have the calls, but a phone call provides higher bandwidth conversation for those who can join and have an hour and might not have several hours to invest into Talk threads. The calls are more about project management & updates, not making decisions for the community.

Actually, I didn’t come to a leadership call with some agenda about infrastructure. I just reminded folks that we need to find a way to grow the team & resources and that lead people to brainstorming on how that could happen.

FYI – Maurya is actually trying to help understand and document our help desk processes, not infrastructure support.

100% agree; however, we do need to find ways for volunteers to help. How does someone go from new to trusted? How do we know when someone is trusted? Aren’t there some things a new volunteer could help with along that path? Perhaps that’s some fodder for developing OMRS Stages.

If we can onboard people asynchronously, that’s great. As I mentioned here, we need to create some documentation for onboarding people. Perhaps onboarding without a call would help lead to some of that documentation getting created.

FYI – I’m trying to increase resources (people and hardware) for the infrastructure team. The minutes from the leadership call just reflect people brainstorming on how we might be able to better support the infrastructure team, not making decisions in the dark.

As you know, we have 30-40 servers hosted at IU XSEDE We have been talking to groups, including CATIS, on the possibility of helping with support. Nothing has happened beyond us reaching out to them and introducing them to the idea; nothing has been decided. You not out of the loop.

I don’t anyone is suggesting that we just hand out the keys to the kingdom to whoever shows interest, but we do need to find a way to grow the team, which will include finding a way in which people can get involved and gain trust.

The suggestion here was that, if we are onboarding more than just Pascal, a call might be more efficient. This was just restating the statement @janflowers already made on Talk.

There’s a good start to onboarding on this page, but the documentation for OpenMRS.org Infrastructure appears outdated. Is there somewhere that someone would know that there’s an Infra chat or infra team structure? Is that in Google Docs somewhere? Or on the wiki? If there are updated docs, then getting those exposed; otherwise, creating/updating the docs that make the infrastructure process less of a black box could be a great outcome of onboarding people (e.g., Maurya, Pascal, etc.).

1 Like
4

We have to trust them. As much as it pains me, there needs to be a call. This is where we differ. There is very little room for error. If people wanna help with the helpdesk, that’s great. Access to servers is granted when we know people better. I will document things.

1 Like
5

As @burke pointed out - “creating/updating the docs that make the infrastructure process less of a black box could be a great outcome of onboarding people”

We find the same thing. When we don’t hire people for a while, all of our orientation documentation gets out of date. When we go through flurry of hiring, it all gets updated it looks pretty good!

It’s not a perfect system, but it does have the benefit of being organic.

Bill

2 Likes
6

Yes, but that trust level has to be established.

Sadly, there’s manual tasks that need to be done (scheduling for on-call, provisioning user accounts, plus explaining things.

Okay, please consult with me before making a change. Being kept out of the loop stresses me out.

Answered above, but yes. I’m willing to train but I expect basic competencies. This is an area with very little room for error.

Currently – we’re just onboarding @pascal. @lluismf’s access would be temporary to investigate the jira performance issues.

[quote=“burke, post:9, topic:7358”] here’s a good start to onboarding on this page, but the documentation for OpenMRS.org Infrastructure appears outdated. Is there somewhere that someone would know that there’s an Infra chat or infra team structure? Is that in Google Docs somewhere? Or on the wiki? If there are updated docs, then getting those exposed; otherwise, creating/updating the docs that make the infrastructure process less of a black box could be a great outcome of onboarding people (e.g., Maurya, Pascal, etc.). [/quote]a

I will be updating the documentation. We’d been winging it thus far, but basically as I see it, we need:

  • A person willing to communicate openly
  • trustworthy (this can be established – I’d say /dev/2 or greater with skillsets we need.
  • able to learn on their feet

Initially, we’d probably have somebody start with helpdesk – triaging actually happens automatically – Desk has great automation for that. If @maurya wants to become a member of the helpdesk – I’d look for proficiency with the command-line.

7

People can start with helpdesk – that’s what we need most. They’d need to give us some stuff they’ve done and show us they have the skills.

This is how Open Source works, things happen in the open. This is one of the things about this community that disturb me. It seems to be run like a business, it’s an open source project.

Yes, but unless people know where to look, it’s not public.

I denied one person direct access because they literally just started coming around. I don’t know them Trust matters. Sorry, disagree here. Infrastructure is a different beast. If things break, our servers become compromised, it’s work for us to clean up. How do we avoid this? By carefully controlling who has access.

8

@r0bby, let me focus on one specific point:

[quote=“r0bby, post:7, topic:7387”] I denied one person direct access because they literally just started coming around. I don’t know them Trust matters. [/quote]

Taking a step back from this, we want to project a posture that is welcoming and open. And when someone expresses the desire to help, our answer should always be “great! let’s figure out how to use your skills and interests!”

If someone says “I can help with JIRA” our answer shouldn’t be “you’re too new to have admin access” even if they are too new. Rather we should say “Great! You can start off by analyzing the problem and suggesting potential solutions.” (We shouldn’t immediately give them admin access, but we also don’t need to mention that if they haven’t asked.)

2 Likes
9

I didn’t posture it that way, in that thread I did make it clear that I wanted their input and they understood where I was coming from.

1 Like
10

I have to agree. The last few projects I have been, I have spend more than 3 months trying to document everything in confluence so I guess one thing to start doing its the documentation. Some might not like doing documents but, if they just do a dump of commands or notes then we can format it correctly. Its better than having nothing.

2 Likes
11

We have a shared Google drive folder

12

Hello @r0bby and everyone concerned,

I wanted to take a look at our infrastructure process from a 3rd person view, like if I want to volunteer do I know what I am volunteering for?. It wasn’t clear to me - what was happening. All I’m doing is seeing, asking questions and documenting. That’s all there is to my involvement. I didn’t think this would hurt anyone. I’m sorry if it stressed you out. It was not my intention.

I believe documenting the effort and type of work will encourage new volunteers as they understand what they are volunteering for.

Then I started looking at people in teams and started reaching out to people to understand what they are doing and how they are using helpdesk. I started with the non technical ones and which I thought would be the easiest to document. As I was reaching out to people I reached out to @janflowers (she handles partnerships) she felt what I was doing to be a good thing to talk about.

I still have to talk to you and @maany and many other people. I am sure it will take me some time to understand what you guys are doing. I am not trying to do infrastructure support, or trying to get a role in infrastructure, I am trying to document the process of what every team is doing in helpdesk.

No one made me do this. No decisions were made.

Here is what I’ve got - https://wiki.openmrs.org/display/RES/Help+Desk. It is not structured and is not complete.

If there is something wrong in it feel free to update it.

2 Likes