Blog by Sumana Harihareswara, Changeset founder
Set Contributor Expectations To Reduce Ghosting
Let's say you're a maintainer who's just received a new contribution. A new contributor has submitted a patch/pull request. And the fundamental idea of it is one you like, but the diff itself will require some more labor to get it good enough to merge.
You know it'll likely take a few back-and-forths for you to explain what they should do and coach them through possibly multiple revisions. And you know it'd take less time for you to fix it yourself (or ignore their work and do it yourself from scratch) than it would to teach them.
You want to help them do it, to give them that great experience of growth, and to grow your contributor base. This is an opportunity to nurture a potential co-maintainer and improve bus factor.
But one reason you feel bleh about the prospect of the iterative review-revise-resubmit process is that you don't know whether they'll vanish and ghost you. You might put in the work to give them constructive and specific feedback, and then get silence in return. And then you'd be left uncertain: do they ever intend to return and finish this work? How long should you you wait before closing it or taking it over? Would you feel like you're stepping on their toes, in case they've started the revision but haven't submitted it yet? You have no way of knowing whether they'll disappear in the middle of your collaboration, and they have no way of knowing that uncertainty about this gives you second thoughts.
Either you make the leap of faith that this stranger will stick around and be responsive for the next few weeks or months of intermittent communication inherent to the process, or you don't. You ignore the patch, or leave it for a "later" that never comes. Or you explicitly say that it's not good enough and you'd rather do it yourself, and close the thread.
Or maybe there's another way.
(This is another post spurred by my conversations with seasoned open source maintainer David Lord. Thanks, David!)
Why this happens
Let's say I send you an email to ask you for something or make a suggestion. To a lot of us, in most of our lives, if you are silent, that's a soft "no." It's a diplomatic, conflict-softening way of indicating that you are not interested.
And so a lot of open source contributors seem to react accordingly, especially if they're new to the project. The more days they have to wait, the greater the chance that they will assume rejection and mentally move on. Some data: the 2014 study "Measuring Engagement: Recommendations from Audit and Analytics" by David Eaves, Adam Lofting, Pierros Papadeas, Peter Loewen of Mozilla, particularly pp. 18, 20 and 26-29 of the slide deck. (Wayback machine capture of context from Mozilla's now-defunct wiki.) Their analysis:
Contributors who received code reviews within 48 hours on their first bug have an exceptionally high rate of returning and contributing.
Contributors who wait longer than 7 days for code review on their first bug have virtually zero percent likelihood of returning.
But maintainers of open source projects know that delay in open source often doesn't mean "no," it means "maybe" or "yes if". Because: something perfect, we can quickly accept. Something utterly terrible or off-base, we can quickly reject. What takes longer than "yes" or "no" is to thoughtfully and diplomatically respond with a probable maybe. Maybe it's the wrong architectural approach, a bad implementation of a good idea. Or: maybe it's ahead of its time, and we know that integrating it properly requires us to implement some other infrastructure first. Or maybe there's something we need to fix privately, like a security vulnerability, before we can publicly discuss this new contributor's idea.
Since maintainers often feel pressed for time, we end up taking days or weeks to say "maybe" or "yes if". By then, they've likely read the signals they think we're sending, and mentally moved on and made alternate plans. So they won't have the time available to task-switch back to this context, revise their contribution, and compose a well-written note about their change. And, unless you've taken care to be appropriately encouraging in your "please revise and resubmit" note, a longer delay is more likely to make them think "what's the point" and to infer that the project is unenthusiastic about collaborating with them.
In general, different contributors, and different kinds of contributors, often expect different paces and rhythms in open source work. (I discussed this further in my piece on cadence shear, and in a related piece on things we can learn from InnerSource.) The less a specific contributor knows and understands the usual rhythm of your team's work, and the less they feel socially able to escalate or ask "how long is this going to take?" without ruffling feathers, the more likely they are to silently decide this isn't going to work out. So this problem significantly affects your ability to build a more diverse contributor base. People in marginalized groups often learn that they get punished for asking an authority figure what's up. But also, I'm talking about diversity across work cultures -- academics, salespeople, military veterans, entrepreneurs, political activists, and manual laborers are likely going to be used to different power distances and different expectations about when they should speak up, to someone above them in a hierarchy, about something blocking their work.
And even when we do reply, if we're asking them to do more work, that can be a big surprise. Because, especially if they're new to open source and/or haven't been a senior developer on a team before, they often don't realize that submitting a patch or pull request is the start of a collaboration. They often think, basically, that the maintainer is going to either accept or reject the PR -- that's it -- or treat it as raw material for the maintainer to improve.
In other words: can we really say they're ghosting if neither side of the collaboration has a clear, explicit expectation about how responsive the other side is supposed to be?
And: People vanish from volunteer opportunities and obligations in all sorts of settings, outside of open source, in different ways and and for many reasons. I've discussed this in detail in my newsletter; read the section "Helping unreliable colleagues". But the structures and incentives of open source projects are a conveyor belt that turn reliable colleagues into overwhelmed overcommitters. We have ambiguous expectations about who's in charge of what, how fast things should happen, what constitutes a commitment, and so on.
What you can do
You can prevent misunderstandings, get better at setting reasonable expectations, and use brief, inviting follow-up notes to encourage them.
Articulate your vision!
When someone suggests an idea that you feel uneasy about, sometimes it's because it wouldn't fit with your vision for the project, and you realize you've never actually written down and published that vision, so they had no way of knowing it wouldn't fit and so rejecting it feels unfair.
So, fix that. Try to write down a few paragraphs about your goals for this project, and about your usage assumptions. Here's an example from the Zulip project saying that it is a real-time chat application, that one server can host multiple realms, and so on. (Comparing the version I originally wrote in 2016, really very little has changed.)
Along the way, try to write down your "non-goals" or "a to-don't list". If you publicly document the kinds of features you aren't interested in adding, and why, that makes it easier for potential contributors to avoid bothering you with ideas you'll have to reject. And even if they don't read it and they suggest something unsuitable anyway, they'll feel less personally rejected if your reply can include a link to your already-published "non-goals." The implication is: that isn't the sort of thing we're interested in, but this is nothing personal; no offense.
If you're having trouble figuring out what your goals or vision are, or your usage assumptions boil down to "for everyone in every circumstance," try updating your written roadmap to highlight your priorities for the upcoming release and the one after that (here's what I think a good roadmap does and how). Or, try using extremes or contrasts to jog your intuition. There are 8+ billion people in the world; whom is this not for? What large changes in the world as a whole, in computing, or in competing tools would make it so that your project would not be necessary? Maybe that can help you set some limits and goals, like:
TalonStick is a jewelry design application for parents and children to use together. It creates design files that can be 3-D printed on mainstream consumer 3-D printers, and renders image files that can be easily reused on the web, as well as asset objects that can be sold and reused in digital games. The application is a native app for Android phones and tablets, using a stylus or fingertip for interaction.
Supported languages: English and the languages that volunteers translate 100% of strings for.
Supported hardware: We aim to be backwards compatible to Android devices that were being sold new by retailers in major world cities within the past five years. If we had significantly more funding, we would provide a mode or support a fork that could be backwards-compatible to significantly older phones and tablets.
Accessibility: We aim to be accessible to limited-vision and color-blind users, but do not aim to provide an accessible experience to users who have no vision.
Privacy: Because children's safety is so important, we do not run a cloud service that interacts with the app, and aim to avoid providing pathways for children to share their information online. This means that we will not provide direct "upload your design to such-and-such game world" functionality, recommend users' creations to each other, etc.
(I made up TalonStick as a fictional example; sorry it isn't as funny as WhatTimeIsIt.com from Joel Spolsky's 23-year-old blog posts about functional specifications.)
If you help people understand what your design and goals aim for, and what and whom they exclude, you'll help everyone make better decisions.
Tell them in the contributors' guide.
David realized as we were talking:
The contributing guide we have right now, and I think most contributing guides that I can think of, are pretty short, and they cut off right after "Okay, you've decided to work on the thing. You did the work and now you made the PR." And that's it. Like, there's no expectation: "Okay, now we're going to collaborate with you."
Let's fix that!
As a comparison: more and more job listings these days have a section on "what to expect from our interview process" (example from Recurse Center). They explain that, after you submit an application, RC will tell you within a week whether they'd like to proceed with the next step, which is a call lasting about 30-45 minutes. And so on, including that RC will "let you know within two weeks of your interview whether or not we’re making an offer". Another example: Latacora's careers page and their current opening for an IT Security Engineer detail an evaluation (work sample) task and how long it ought to take the candidate.
This gives a potential applicant a sense of what the process is going to be like ahead of time, including how long they should expect to wait for decisions. Candidates can decide for themselves ahead of time whether they could stand to wait that long or go through the number of interviews and tasks required. (I suspect that pre-committing to a process also has a side effect of forcing firms to keep to more reasonable timelines, and force jobseekers to jump through fewer hoops.)
Similarly, potential open source contributors often need you to explicitly set expectations about how long it will take you to review their work, and what you will likely need from them along the way, and on what timescale and cadence.
Also: remind them to check for updates. For instance, if you have automated tests set up to run for every patch, do contributors get notified of success or failure? Will they need to manually come back and check an hour after submission? (If so, maybe you ought to to set up automated notifications of failures.) And if they have notification emails turned off on GitHub/GitLab/Launchpad/Codeberg/Sourcehut/etc., or sent to a secondary address, they need to address that.
Here's an example of text you could add to the end of your contributor guide:
After submission
Our goal is to work with you so that you get to refine your patch and make it really outstanding, and then we can merge it in. So, over the next few weeks, we'll request that you improve this patch in specific ways, and the way we'll tell you is through [NAME OF PLATFORM] comments and code reviews. So please make sure [NAME OF PLATFORM] notification emails aren't going to your spam folder. And, if you won't be able to address those comments within a couple of days, just comment with a brief note saying that, so we know you haven't disappeared completely, and we can plan our work accordingly.
If you end up not responding for a while, we understand, but also we may close this thread in about a month and move on with other things. Because the rest of the codebase will continue to change, it's hard for us to revisit a patch months later and work with you to get it rebased and merged in. Or, if you aren't responding to our questions and suggestions, we may take over your patch, improve it ourselves, and merge it, with you listed as one of the co-authors.
After you submit a patch, you can expect us to give you some initial review comments within the next 2 weeks. And then, once we're collaborating with you on it, you can expect replies to your follow-ups within a few business days.
Auto-reply or boilerplate reply to set expectations.
When we want to invite people to hang out with us, but we haven't yet figured out the particular time or other details, we offer them a heads-up, a preliminary invitation. The most formal version of this I usually see is a Save-The-Date card in my paper mail, promising a fuller wedding invitation to follow. More frequently, a friend might say, "Are you free Thursday night? It'd be great to see you, if that works for you." I should answer them, but also I should lean towards not making a conflicting plan.
You can do this with new contributions, too. Your first reply doesn't need to address everything in the patch. It can say something like this:
Hi! Thanks for this. Our goal is to work with you so that you get to refine your pull request and make it really outstanding, and then we can merge it in. So please check out the "after submission" section of our contributor guide [LINK] and follow those instructions to make sure the next few weeks go smoothly.
Looking forward to working with you! You can expect us to give you some initial review comments within the next 2 weeks. Thanks again.
You can store this up as a "saved reply" in your GitHub settings and use it that way. You could also use a bot such as Zulipbot or Carsonbot to automatically reply with this text if a user submits their first patch or issue to your repository or organization. Even better: customize it with special instructions for a person's first-ever contribution on the platform you're using (such as Sourcehut or GitHub).
Even if you put guidance about this in your contributor guide, let's acknowledge that many contributors skim or skip detailed contributor guides. So it's a good idea to share this info at the moment they're most primed to notice and read it: right after they make a contribution.
Prioritize quick first-level reviews.
When you're a maintainer who wants to retain new contributors, it's key to reply to new project-related emails, queries, bug reports, and patches fast. Reply to people fast, even if your message is a quick "I saw this, thank you, I'm busy, will get to this in a few weeks," because otherwise the uncertainty is deathly and people's enthusiasm and momentum drip away. Even an "I won't have time to review this for several months" reply is a useful signal that communicates: you value that they contributed, you respect their time, and you have nonzero interest in collaborating with them.
You might think it's better to wait till you have time to maximize how optimistic and positive you can be, salvaging elements to praise. But, as the Mozilla analysis indicates, a kind, fast, negative response is better than a long silent delay, even if that delay would be followed by a more positive response. Prioritize getting back to people fast, to signal that you genuinely want to work with them, and you'll have an easier time eventually asking them to come back with a better patch.
As Sage Sharp discusses, consider code review as taking place at 3 levels. You're asking:
First: Is the idea behind the contribution sound?
Second: Is the contribution architected correctly?
Third: Is the contribution polished?
You can often respond on that first level quite quickly -- you could say, "Thanks! I like the idea of this change; I'm going to need a few more days to look at it in depth." Or: "Thanks for the patch. I'm not sure about whether this works with the design of the project as a whole; I'll think about this and get back to you with some more detailed thoughts in a week or so."
If the contribution clearly and obviously passes the first level of review, but you don't have time to do an architectural review right away, this may be an opportunity to delegate. Ask a non-maintainer contributor to take a look and share their opinions. This is also great as a step to help them gain experience in case you'd someday like to promote them to co-maintainer.
If a skim doesn't give you a sense of the fundamental idea behind the contribution, you can reply to tell them you're having trouble understanding it and ask them to write a short summary to help you understand their goal. (Leave a comment if you'd like some diplomatic language you can copy!)
My friend Jacob mentioned that, when he has an idea for a change to a FLOSS project, sometimes he contributes a hasty example patch -- working code, but rough and without tests -- and asks, basically, "am I on the right path? is this even something you would consider?" He's basically handing the maintainer a golden opportunity to save everyone time. Some maintainers reply within a week or two, but many don't. If they don't reply at all, he treats that as though they have said they aren't interested, but he is also less interested in trying to collaborate with those projects in the future at all. In contrast, if they explicitly reply to say no, he can happily close that thread, and retains his interest in contributing to the project if it makes sense someday.
Follow up with "resources in case you're stuck" checkins.
Sometimes, when you do not follow up with a new contributor who has gone silent, they infer from your silence that you are uninterested in their contribution and aren't interested in helping them. Don't underestimate the power of invitation here. If it's been two weeks since the last interaction, even if maintainers were the last ones to initiate contact, reply to their patch with a note reminding them that they can get help if they're stuck. Example:
Hey - in case you're having trouble with this, please do let us know and ask for help if you need it! You might find some useful tips at [LINK TK], and you can talk with us in [CHAT CHANNEL/FORUM].
Again, something you can at least partially automate with a bot -- or you can use search queries to help you find people who haven't replied for a while to maintainers' questions and who might benefit from a nudge.
Build safeguards against cookie-licking.
Karl Fogel says: "The theory should always be that any developer is welcome to help out on any task at any time" and shares ways to avoid territoriality and elitism in your project in his book Producing Open Source Software. Your openness lets any new contributor pop up and say "I'd like to work on this issue." And that's great!
But if they take a long time to follow up, they may accidentally "lick the cookie," which is annoying and slows down others' work. And that prospect may make you grumpy and reduce your motivation to be as welcoming as you want to.
"Cookie-licking" is when someone says, in public, that they're going to work on something. It's like licking a cookie on a shared platter to mark a claim. Other people, who don't want to disrespect someone else's commitment, trespass their turf, or duplicate their effort, refrain from working on that thing. And in the culture of open source, where we prize autonomy and don't want to micromanage each other, we often avoid explicitly asking: "Are you still working on this? When do you think you'll be done? Might it be better for you to let someone else do it instead?"
(I know I didn't invent this term but am not sure who did. It's been on a community management wiki for years; Raymond Chen in 2009 had the impression it was in circulation at Microsoft.)
Cookie-licking is a dynamic that emerges in lots of collaborative situations, and it's normal. You can mitigate it. Two approaches that I've seen work:
- Automatically un-assign issues that go quiet. Zulipbot's "Track inactive claimed issues" functionality does this: "If a claimed issue has not been updated for a week, @zulipbot will post a comment on the inactive issue to ask the assignee(s) if they are still working on the issue.....If @zulipbot does not receive a response from the assignee within 3 days of an inactive issue prompt, @zulipbot will automatically remove the issue’s current assignee(s) and the 'in progress' label to allow others to work on an inactive issue." This helps build a norm that a contributor should frequently update a thread with progress reports, encouraging them to act more like Sloppy Sam than like Perfectionist Pat. And it indirectly encourages contributors to only claim a few tasks at a time.
- Explicitly distinguish fuzzy hopes and committed plans. In Wikimedia tech, people try to work in public a lot. For some people, that includes maintaining aspirational, public "TODO" lists, or chiming into email threads with "I have some ideas about how to do that" notes. Back in 2012 while working at the Wikimedia Foundation, I witnessed a few instances where this led to accidental cookie-licking, and sent the following email:
In my household we have a word for "sort of a plan but nothing definite and still in flux". It is a plin.
A plin turns into a plan once you've made a real commitment that has a cost. For example, a vacation plin turns into a plan when I buy airplane tickets.
I think the equivalent of an expensive commitment in our community is a public statement like "I will spend the next week working on $foo" or "I will deliver $bar by the end of the month" or even "I am actively working on $baz and will review new contributors' code in a timely fashion". That is more like a plan.
Plins are good! We need brainstorming and good intentions. But they are not really plans.
I agree with Mark about the danger of cookie-licking. BE BOLD (and loud) in overriding other people's plins with your own efforts!
This did make a difference!
Feel free to borrow the term "plin." And, more generally, I recommend you encourage experienced contributors who are claiming tasks to publicly make expensive commitments, and I encourage you to set up norms where someone willing to make an expensive commitment can override someone who's only made a plin. If you know that's okay, then you'll likely feel less resentment of occasional contributors who have accidentally licked cookies.
Other kinds of ghosting
In this post I've only covered one common reason volunteers vanish. Some others include:
- Someone's being a jerk.
- I got pushback I didn't expect, and realized you and I are on completely different pages regarding our goals for the project; rather than argue, I found it easier to just step away.
- I got stuck and don't know how to make progress on this task, and that's overwhelmed me.
(Want me to write about any of these? Let me know in the comments or in the Fediverse.)
Zooming out
On one level, you're making a piece of software, and on another level, you also need to maintain and likely improve the infrastructure that helps you make and distribute that software. And that infrastructure can be
- digital -- such as build automation, testing, and bug tracking
- social -- such as procedures for making decisions, and your users' ability to tell you things and hear from you
- financial or legal -- including money, and a legal entity that makes it easy for you to raise and spend it
My advice above tells you ways to improve your digital and social infrastructure to address a particular kind of problem. When you run into a gnarly problem over and over in your open source project, try thinking about those four kinds of infrastructure, and consider whether improving one of them could help.
(Much more forthcoming in my book.)
