Blog by Sumana Harihareswara, Changeset founder

27 Jan 2021, 7:01 a.m.

Gaps in Existing Guidance on Open Source & Software Management

Getting Unstuck sampler cover I'm working on a book proposal for the full-length book version of Getting Unstuck: Advice for Open Source Projects (38-page sampler available now for free download when you subscribe to Changeset Consulting's email newsletter (1-10 updates per year).)

In the process, I've developed a list of existing books and online resources on open source maintainership and on software management, and I've thought more about why general software management advice -- which usually assumes you and your colleagues all work for the same company or other organization -- doesn't address most open source maintainers' needs.

Writing a book proposal: kind of necessary, kind of a drag

In nonfiction book publishing, a book proposal is the way you say to a publisher, "I think it would be a good deal for both of us if you published this book I'm writing." For example, No Starch Press would like a summary, an outline, and descriptions of the audience, the competition, the market, and the author. All the publishers I'm considering want to know those things, and some also want to know the schedule for completion, which other books from that publisher cover related or similar topics, the authorial approach I'll use, what keywords readers would search for to find this book, and more.

I might not go with a traditional publisher; I might self-publish, either all at once or in stages, such as via my email newsletter. But writing this proposal gives me more options and makes me think through what I'm planning to do. Still, it can be a drag to spend time on persuading other people that something is a good idea instead of executing on the idea itself, and it is a specific drag for me to spend time writing something that very few people will see.

The most immediately-useful-to-others part is probably the literature review, the overview of books and similar resources that are in any way comparable. Thus:

My competition

So here's my competition. I don't mean to disrespect any of these works or their authors, just to clearly state what they do and what they don't do (and thus why there is still a need for my book).

I hired Audrey Eschright to start this and then I continued it myself -- thanks, Audrey! I know this is incomplete and I'll remember another thing three hours after I post this (I may edit to add things), but I figure it might be useful to folks looking for books and web curricula about open source, project management, maintaining legacy systems, and so on.

Open source & related

  • Book and online resource: Producing Open Source Software: How to Run a Successful Free Software Project by Karl Fogel, 9780596007591, published by O'Reilly, 2005 (web version is second edition, revised 2017)

    Audience: FLOSS newbies, programmers and managers, new project contributors and maintainers

    Covers/teaches: FLOSS culture, communication tools and techniques, project management, governance, structured collaboration

    Good: Friendly, detailed, aimed at software professionals at least as much as side-project folks

    Missing/needs (or, not designed to cover): Mostly about greenfield projects; does not cover how to come into an existing project and lead it (Fogel has thus encouraged my own project as a kind of sequel to his); limited discussion of managing problem contributors, bug triage, and planning roadmap; very little/no discussion of budgeting, grant proposals, and succession planning; somewhat out-of-date regarding discussing modern tooling

  • Book: Forge Your Future with Open Source by VM Brasseur, 9781680503012, published by Pragmatic Bookshelf, 2018

    Audience: FLOSS newbies/new contributors (professional and volunteer), programmers and other relevant skill sets

    Covers/teaches: FLOSS culture, common tools and practices, open-sourcing a personal project

    Good: Reasonable sequence; clearly written; covers major gotchas new contributors will run into

    Missing/needs (or, not designed to cover): Maintainer skills, working with a legacy project

  • Book: Working in Public: The Making and Maintenance of Open Source Software by Nadia Eghbal, 0578675862, published by Stripe Press, 2020

    Audience: Technologists, managers, and interested bystanders such as sociologists and economists

    Covers/teaches: General principles regarding FLOSS sustainability and dynamics

    Good: “An anthropological dive into the stories of real developers” with some useful explanations of some dynamics in open source projects (such as the categories of "toy, club, federation, stadium," based on the ratios of users and contributors and user growth and contributor growth)

    Missing/needs (or, not designed to cover): Is inaccurate in ways that accumulate page by page. This surprised me badly, especially since her Roads and Bridges report was so helpful. My copy of Working in Public has corrections and rejoinders from me in the margins of about every fourth page. I need to write a more thorough review but, in short, Eghbal's choices to ignore all projects that don't use GitHub, and all contribution types other than code commits, and all of the effects of venture capital on the economics of open source, and the roles governments can play in funding digital infrastructure, constrict the analysis and recommendations towards a disappointing conclusion that will not help most of the infrastructural open source projects (and their maintainers) I know and work with. Also, it does not provide practical advice for maintainers (such as instructions and exercises) on dealing with specific issues to rescue legacy projects. (Updated this summary in July 2021.)

  • Online resource: Mozilla Open Leadership Training Series (online curriculum), Mozilla contributors, started in 2016 and updated since then

    Audience: “Anyone starting up or leading open projects– project leads, collaborators, or small groups of co-leaders responsible for project success and growth”

    Covers/teaches: Communication, community-building, using GitHub, mentoring, project maintenance, organizing events

    Good: Multiple kinds of content, exercises to use, explains FOSS collaboration without requiring a technical background

    Missing/needs (or, not designed to cover): Budgeting, working with a legacy project instead of starting a new project

  • [Edited 2021-02-05 to add] Online Resource: TODO Group guides (online), various authors, started in 2019

    Audience: engineering executives at large organizations

    Skills taught: participating in open source communities, building leadership in a FLOSS project and improving open source development impact, and related topics

    Covers/teaches: General overview of how to start participating in open source projects and grow into leadership; realistic and business-focused understanding that contributing to existing projects often has greater return on investment than starting new ones; specific tool recommendations

    Missing/needs (or, not designed to cover): Detailed instructions on providing coordination and maintainership; assumes incoming participants have an architectural/feature agenda (rather than just wanting more frequent releases); lack of detailed examples, exercises; assumes reader already has basic project management skills (since it is aimed at executives rather than individual contributors)

  • [Edited 2021-07-13 to add] Online Resources: "Growing Open Source Projects with a Stable Foundation" by Martin Michlmayr, and the accompanying research report on "the operations and challenges FOSS foundations face", published 2021-04-28

    Audience: Practitioners and funders in open source software

    Skills taught: "This primer covers non-technical aspects that the majority of projects will have to consider at some point. It also explains how FOSS foundations can help projects grow and succeed." Like a shorter, high-level and up-to-date version of Karl Fogel's Producing Open Source Software, focusing on governance and related systems and processes.

    Covers/teaches: FLOSS governance approaches, legal and financial issues, structures to accommodate and encourage project team growth

    Good: Very recent, has lots of examples of how existing organizations address certain kinds of needs

    Missing/needs (or, not designed to cover): examples, techniques, and exercises for learning the skills to implement Michlmayr's suggestions

  • [Edited 2021-02-10 to add] Online Resource: The Open Source Way (online), various authors, first edition 2009, second edition 2020

    Audience: People, with varying levels of experience participating in open source projects, who want to participate in and lead them

    Skills taught: Participating in open source communities, building leadership in a FLOSS project and improving open source development impact, and related topics

    Covers/teaches: General overview of how to start participating in open source projects and improve their user and participant bases, and measure one's success; realistic advice based on experience

    Missing/needs (or, not designed to cover): Mostly about greenfield projects; does not cover how to come into an existing project and lead it; limited discussion of managing problem contributors, bug triage, and planning roadmap; very little/no discussion of budgeting, grant proposals, and succession planning

  • Online resource: (online curriculum), GitHub team and other contributors, started in 2016

    Audience: GitHub users, maintainers of new projects, FLOSS newbies, people outside existing FLOSS organizations, programmers

    Covers/teaches: FOSS culture, technical management, outreach, inclusive practices, leadership skills

    Good: Overview of many important aspects of running an open source project

    Missing/needs (or, not designed to cover): How to turn around a legacy project, especially when starting as a non-maintainer contributor; non-GitHub forges (resource is entirely GitHub-specific)

  • Online resource: Google Summer of Code Mentor Guide , multiple authors, started in 2009 and updated since then

    Audience: FOSS project mentors

    Covers/teaches: Mentoring, communication, FOSS cultural knowledge, commit/patch management

    Good: Detailed guidelines for mentor/student interaction and evaluating progress

    Missing/needs (or, not designed to cover): Organization could be improved; indicates but doesn’t address the issue of project culture not being friendly to newcomers aside from how you guide the student through it; no guidance for how to run a project overall, since it concentrates on just the act of mentoring an intern.

  • Online resource: The Field Guide to Open Source in the Newsroom, OpenNews contributors (including me, Sumana Harihareswara), started in 2016 and updated since then

    Audience: Journalists and news organizations

    Covers/teaches: FOSS culture, tech skills, documentation, community management, leadership transitions, starting a new project as open source or opening the source code to an existing project

    Good: “Handoffs and Sunsets” section, good examination of the beginning and ends of projects

    Missing/needs (or, not designed to cover): Is still unfinished and lacks thorough examples. Does not cover the case of coming into and reviving an existing open source project, and has little to no advice on project management

  • Book: The Art of Community: Building the New Age of Participation, by Jono Bacon, 9781449312060, published by O'Reilly, 2012

    Audience: Novice contributors/leaders/community managers for open source software and open culture organizations (assumption that reader does not already know the basics of how open source contribution works)

    Covers/teaches: Making a mission statement and strategic plan, introducing process, marketing, governance

    Good: Specific advice on communication channels, writing skills, governance, conflict management, basic event planning, and marketing; anecdotes and lessons learned from many projects. Online component.

    Missing/needs (or, not designed to cover): Is at least eight years out of date (example: Gobby and recommendations); assumes you can build new processes and communities from scratch, instead of helping people who are joining midstream, quirky writing style will put off some readers

  • Book: People Powered, by Jono Bacon, 9781400214884, published by HarperCollins Leadership, 2019

    Audience: Executives at businesses

    Covers/teaches: Initiating and exploiting user groups for products and services (not specific to open source software)

    Good: Specific instructions on sequence and strategy for creating new user groups

    Missing/needs (or, not designed to cover): No discussions of joining and co-maintaining existing open source projects; aimed at existing executives who are not yet open community contributors, not current open source contributors

Software management

(There are dozens of reasonably well-regarded books on software management in general, from classics like DeMarco & Lister's Peopleware to more recent works like the Fournier I mention below; most of them are only partially suitable for my target audience for the reasons I mention in "What doesn't get covered".)

  • Book: A Civic Technologist’s Practice Guide by Cyd Harrell, 978-1735286501, self-published, 2020

    Audience: People who are doing, or want to do, civic technology -- developers, entrepreneurs, and people in related fields

    Covers/teaches: How government tech works, choosing contribution and project types, ways to avoid common gotchas to make long-term change without burning out

    Good: Distills a wealth of experience, clearly written, realistic, good sequencing

    Missing/needs (or, not designed to cover): Has only a single section within a single chapter on “Open-source teams and assumptions”; generally assumes institutional funding 

  • Book: Kill It with Fire: Manage Aging Computer Systems (and Future Proof Modern Ones) by Marianne Bellotti, 9781718501188, forthcoming to be published by No Starch Press, March 2021

    Audience: Managers within existing large organizations

    Covers/teaches: “How to evaluate existing architecture, create upgrade plans, and handle communication structures.”

    Good: Probably engagingly written based on Bellotti's other work; “team exercises and historical analyses of complex computer systems”

    Missing/needs (or, not designed to cover): Assumes that the project is housed within a single organization, and is thus mostly inapplicable to multi-stakeholder projects such as volunteer-run open source projects

  • Book: The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change by Camille Fournier, 1491973897 , published by O'Reilly, 2017

    Audience: Engineering managers within companies and similar organizations

    Covers/teaches: Leadership, planning, decision-making, team culture, people management

    Good: Coverage of common dysfunctions, good sequencing and clear writing; covers both line-level and senior leadership roles

    Missing/needs (or, not designed to cover): Very little attention paid to open source dynamics; Assumes that the project is housed within a single organization, and is thus mostly inapplicable to multi-stakeholder projects such as volunteer-run open source projects

  • Book: Making Things Happen: Mastering Project Management by Scott Berkun, 9780596517717, published by O'Reilly, 2008 (formerly “The Art of Project Management”)

    Audience: Project managers at large organizations

    Covers/teaches: Leadership, planning, decision-making

    Good: Detailed, funny, includes guidance on communication methods (such as meetings and emails) as well as discussion questions

    Missing/needs (or, not designed to cover): Assumes that the project is housed within a single organization, and is thus mostly inapplicable to multi-stakeholder projects; assumes in-person collaboration and does not accommodate open source project approaches

If I've made any substantial errors in my descriptions of these books and websites, please let me know. And if you think I've missed a work, if you think the book I'm working on substantially overlaps with prior work that I have not mentioned, please tell me. I don't want to waste anyone's time and I wish to minimize duplication of effort.

What doesn't get covered

There are lots of guides to starting open source projects, but overall they do not address the needs of a new maintainer of a legacy project. As Marco Rogers recently observed regarding code-related tutorials: "there is very little content that is appropriately labeled as intermediate to advanced....A lot of content is pointed towards either newbies or people doing greenfield work."

And there are many books about managing software projects, including complex infrastructural and legacy projects. They generally assume you're making proprietary software, and so (except for works like Bacon's) they don't account for the benefits of working in the open, the possibility of getting gratis contributions from users, open source strategy for the enterprise, and so on.

But also -- more crucially for a project management book -- on the whole they assume that all contributors are paid staffers, usually of the same organization. This is a somewhat less obvious distinction so I'll discuss it at a bit more length.

The job of a project manager varies wildly depending on how much power you actually have to say no to things and change delivery deadlines, whether you have the power to hire and fire people, and whether the colleagues who work on your project are solely working on your project (or splitting their time among multiple projects). Much of the existing writing on software management assumes that you are working in a mostly-hierarchical environment bounded by a single organization, where someone has the power to hire and fire, there is a monetary budget you control or have to keep track of, and so on.

Certainly some orgs are more hierarchical than others and there exist some where you basically have to use persuasion if you want a change to happen. First, of course that dynamic privileges some people, and it's worth checking for -isms in who gets to just veto things for no reason and who doesn't. And second, even so, if you and the other people you are influencing are in the same org and are being paid by the same employer, you still have different cues and levers available to you. Here are some structural differences between managing a more cross-org or extra-institutional project and managing one where everyone is being paid by the same employer:

  • If you send an email or assign an issue, they are more likely to feel pressure to actually read and answer it; at some point, if they absolutely never listen to or respond to stuff their colleague is saying, that may lead to repercussions in the rest of their job. Ditto for if you invite them to meetings, retreats, etc.
  • It's much more possible to make systematic changes that then affect lots of projects/individual contributors through systematic incentive or environmental nudges that apply to ALL contributors, e.g., working with senior managers to make [activity] a factor in promotions, which then facilitates getting [activity] to happen in YOUR project(s).
  • If your team is remote, it's somewhat easier to arrange an in-person retreat, because it's way easier to get and figure out budget, it's easier to figure out *whom to invite*, to *schedule* it, to deal with expenses and so on. This means you can have more and better high-bandwidth meetings to build trust and make complicated decisions.
  • The existence of centralized IT that you (and individual contributors) probably don't have to run yourself, and that everyone already is accustomed to using, and forces that somewhat delay/prevent tool fragmentation, means that if you set up a project management dashboard or similar, it's easier to ensure that all your colleagues are getting reports from it, that it ties into their existing communication feeds, and so on.
  • You likely can access a list of your colleagues and (in organizations that are not giant) you can find out a fair amount about them when a new one comes onboard in a role that you will interact with, and you can be alerted or find out when one leaves the org entirely.
  • There is a more bounded, finite set of stakeholders for you to deal with (including whom you could possibly ask for more budget), and it's easier for you to know who your users are.
  • If someone really does not want to obey the Code of Conduct, it can escalate to the personnel department and they may get fired.

Thus: my book

So I am continuing to work on the full-length book version of Getting Unstuck: Advice for Open Source Projects (38-page sampler available now for free download when you subscribe to Changeset Consulting's email newsletter (1-10 updates per year).)

Once I finish this proposal.