Research and Nonprofit Software Debt: Grants, Turnover, and Reproducibility
Last updated . Sources are named and dated inline - how we source claims.
Academic labs, research groups, and nonprofits run some of the most consequential software in the world on funding models that pay for novelty and almost never pay for maintenance. The debt that results is structural, not careless.
Grant cycles, doctoral turnover, reproducibility, domain experts writing production code, and infrastructure that began as a one-off script all interact here in ways they do not in commercial software.
The Incentives Are the Problem, Not the People
Most writing about research code treats scientists as people who could write maintainable software but cannot be bothered, and then offers them a lecture about unit tests. That framing is wrong on the facts, and it is why the advice bounces off.
Consider what a research group is optimizing for. Funding arrives in fixed-term awards attached to novel questions. Careers advance on publications, and the currency of a publication is a result. Positions are temporary by design, cycling on a schedule set by the funding rather than the software. In that system, an hour spent making code maintainable is an hour not spent on the thing that is measured, by a person whose position may end before the maintenance pays off, on a project whose funding did not include maintenance as a line item.
A researcher who writes quick, undocumented code there is responding correctly to the incentives they were given. The useful question is not "how do we get researchers to care about software quality" -- many already do -- but which small number of practices survive contact with these incentives.
Grant-Funded Development: Novelty Is Funded, Maintenance Is Not
Research funding is awarded for work that is new. That is the point of it. The side effect is that sustainability is structurally unfunded: there is a mechanism for building a tool and none for keeping it working. The resulting pattern is recognizable across every field. A grant funds a project, the project produces a tool because the science required one, other groups adopt it, and then the grant ends. The tool is now depended upon by people who did not build it, maintained by nobody, and hosted on a server whose renewal is somebody's personal problem.
A second effect is subtler. Because maintenance cannot be funded directly, it gets smuggled into the next grant as a fraction of somebody's time on a project nominally about something else. That is how a great deal of important research software stays alive, and it makes maintenance invisible, unplanned, and permanently in tension with the deliverable that justified the money.
There are workable responses inside the constraint. Write software effort into proposals explicitly rather than assuming it will be absorbed, and treat sustainability or infrastructure calls as a real category of proposal. Design tools so the expensive parts are separable from the novel parts, letting a successor adopt a component without an entire codebase. And be honest about support status -- a clearly labelled unmaintained tool is far more useful to a downstream researcher than one that looks alive and is not.
The Turnover Clock: Knowledge Loss on a Predictable Cycle
In a company, losing the only person who understands a system is an incident. In a research group it is the schedule. Doctoral programmes run a few years, postdoctoral contracts are shorter, and every one of those people leaves by design on a date known when they arrived. The software they wrote stays.
What makes this different from ordinary bus-factor risk is that it is periodic and guaranteed. A lab knows with certainty that its most knowledgeable contributor will be gone within a fixed window, and that whoever inherits the code will be new to the field as well as the codebase. The result is a lab where three generations of analysis pipeline coexist, each written by someone who could not fully understand the previous one, and where the honest answer to "why does this step exist" is that a former student added it and nobody dares remove it.
The fitting mitigation is to treat handover as a scheduled event rather than a courtesy, with the group agreeing in advance what it means: the environment rebuilds from a file in the repository, someone other than the author has run the pipeline on a fresh machine, and the README explains what the code is for. The cheapest version is the highest value -- have the incoming person run the pipeline while the outgoing person can still answer questions. That single exercise finds the undocumented environment assumption, the hardcoded home directory, and the manual step nobody remembered was manual. See training and culture for making knowledge outlive the people who hold it.
Reproducibility Is a Debt Problem
Everywhere else, technical debt makes software slower to change. Here it can make a result impossible to verify -- and in research, a wrong answer is far worse than a slow one.
Undocumented Environments
The analysis ran on a machine with years of accumulated manual installs. Nothing recorded which library versions were present. The environment was a fact about one computer, and that computer has since been reimaged.
Unpinned Dependencies
A requirements file naming packages without versions installs something different every year. Numerical libraries change defaults between releases, so the same code can produce a slightly different number -- the dangerous kind, because nobody notices.
Works on the Cluster
Code depending on a shared filesystem layout, a site-specific module system, or a data directory existing at one institution only. Entirely correct and entirely unrunnable anywhere else, so nobody outside the group can check it.
The Undocumented Manual Step
Between stage two and stage three, someone opens a spreadsheet and removes the outliers. It is not in the pipeline, not in the methods section, and not in anyone's memory a year later. Manual steps leave no artifact at all.
Uncontrolled Randomness
Unseeded random number generators, nondeterministic parallel reductions, and hash ordering that varies between runs. The result is reproducible only in distribution, which is a serious problem if the paper reports a single number.
Untracked Input Data
The pipeline reads a file edited in place several times, or pulls from an upstream database since revised. Without a checksum or a version, the code can be perfectly preserved and the analysis still cannot be repeated.
This is why the same debt costs more here than in industry. When an e-commerce codebase is a mess, the product ships late. When an analysis pipeline is a mess, a published result may be unverifiable, and a mistake inside it can survive into other people's work through citation. The cost is paid in the reliability of the literature.
It also explains why reproducibility practices are the easiest software engineering to justify here: pinned environments, recorded data versions, and a scripted pipeline are not code quality improvements, they are experimental method. Framed that way they belong in the methods section, and they are defensible to a reviewer in a way "refactoring" never is. Versioning data and tracking provenance overlap heavily with machine learning and data debt.
Code Written by Domain Experts
A great deal of research software is written by people with deep expertise in a scientific domain and no formal training in software engineering, because none was offered. A biologist or a climate scientist learns enough programming to answer their question, largely by imitation, and gets remarkably far. The code has a characteristic shape: long procedural scripts, copy-and-paste variants per dataset, few functions, no tests. It is worth saying clearly that this code frequently works, and that the domain knowledge encoded in it is the hard part -- an engineer parachuted in can restructure it into something tidy and silently break a correction factor they did not understand.
So the way to help is not to lead with a critique of the code, but with a problem the researcher already has: an analysis that takes a week to rerun, a result that could not be reproduced, a reviewer asking for a sensitivity analysis that means changing a parameter in fourteen places. Fixing that specific pain, with them rather than for them, demonstrates the value better than any argument about maintainability. Pairing helps; rewriting their analysis over a weekend does not, because they can no longer modify it. Introduce one practice at a time, use the vocabulary of experimental method rather than software architecture, and never open with the word "just".
The Script That Became Infrastructure
Nearly every research group has one. It was written in an afternoon to answer a single question and never meant to survive the week. It is now run before every submission, depended on by three other groups, and quietly load-bearing -- still a single file with hardcoded paths, no error handling, and a comment saying it is temporary. Nothing ever announced the transition, so there was never a moment at which anyone reconsidered how it was built.
The practical response is to notice the transition deliberately. Useful signals: someone other than the author is running it, it runs on a schedule, a published result depends on it, or its failure would block other people's work. Any one of those is the moment to promote it -- a repository, a pinned environment, a README, and one test proving it still produces the expected output. Promotion does not have to mean rewriting; most of the value comes from making the thing reproducible and legible rather than elegant.
Research Software Engineering as a Discipline
The structural answer to a structural problem is a role, and one now exists. Research Software Engineering has emerged as a recognized discipline for people combining software engineering skill with enough domain knowledge to work inside a research group -- a career path that previously did not exist, which meant anyone drifting toward it was penalized by a promotion system counting only papers.
The Software Sustainability Institute, based in the United Kingdom, describes itself as "the first organisation in the world that was dedicated to improving software in research," and states its purpose as helping people "build better and more sustainable software to enable world-class research."
In the United States, the US-RSE Association performs a similar function, stating plainly that "Research Software Engineers use expertise in programming to advance research" and describing its aim as building "a connective, supportive, and diverse community of individuals from universities, laboratories, knowledge institutes, companies, and other enterprises."
Institutions are creating these roles because a shared RSE group is the only funding shape that matches the demand. Individual grants cannot sustain a maintainer, but an institution supporting a central team can spread that cost across many projects, retain the expertise between grants, and offer a career track with progression in it. That last part matters more than it sounds: research software knowledge kept evaporating not because of turnover alone, but because someone who was good at this had nowhere to go.
Community standards have appeared alongside the roles. The FAIR4RS Working Group adapted the established FAIR guiding principles for data to software, reporting that it "has adapted the FAIR Guiding Principles to create the FAIR Principles for Research Software (FAIR4RS Principles)" as a basis for discussion of their adoption. These give a group something concrete to aim at rather than an unbounded instruction to write better code.
Citation and Credit: Work With the Incentive
Every attempt to improve research software that fights the incentive system loses. What gets rewarded is papers, citations, and grants; nobody was ever promoted for a tidy repository. The productive move is to route software work into the currency that already counts rather than argue the currency is wrong.
The mechanics exist and are underused. A citation file in the repository tells users exactly how to cite the software, converting downstream use into something that appears in a metrics report. Archiving a release to obtain a persistent identifier makes a specific version citable and guarantees the version behind a published result still exists. Software papers turn a tool into a publication, and a well-used tool can accumulate citations for years after its funding ended.
None of this is cynical. It is the difference between asking someone to do unrewarded work indefinitely and building a path where the work is visible to the people who make funding and hiring decisions. The same logic applies to sourcing generally -- see research and citations for the standard applied to every figure on this site.
Nonprofit Constraints
Nonprofits share the funding shape and add constraints of their own. Restricted funding means money arrives earmarked for programme delivery, and overhead -- which is where software lives -- is exactly the line item donors scrutinize most. An organization can be well funded and still unable to spend anything on the database that runs its operations.
Volunteer contribution is the second constraint. Volunteer developers are generous and genuinely valuable, and also intermittent and gone without notice. Work arrives in bursts from people who cannot be asked to come back and finish it, producing a codebase of half-migrations: two authentication approaches, three date conventions, a partially completed framework upgrade. Each contribution improved something; collectively they left a system nobody has a mental model of. Donated infrastructure is the third: free credits and pro bono hosting are real help, and they create dependencies with expiry dates attached to somebody else's programme.
What works here is ruthless narrowing. Fewer systems, boring and well-supported technology, managed services in preference to anything self-hosted, and a deliberate refusal of complexity a two-person team cannot operate. Scope contributions so a volunteer can finish one in a single session, and for donated platforms know which pieces are portable before the renewal notice arrives.
Pragmatic Minimums That Survive Turnover
Not a maturity model. Three things, chosen because they are cheap enough to actually happen and because each keeps working after the person who set it up has left.
A README That Works
Not a description -- instructions a newcomer can follow on a clean machine to reach a correct result. What the software is for, what it needs, one worked example with expected output, and who to ask. The test is rarely applied: hand it to someone who has never run the code and watch where they get stuck.
A Pinned Environment
An exact, version-locked specification of what the code needs, committed alongside it: a lock file, an environment file, or a container definition. This is the highest-value artifact in research software, because without it the code is only half of the experiment. Record the interpreter version too.
One Test That Proves It Runs
Not a test suite. One end-to-end check that takes a small fixed input, runs the whole pipeline, and compares output to a stored expected result. It catches what matters most here -- a dependency that changed behavior, an environment that no longer builds, a refactor that altered a number.
The reason to stop at three is that longer lists do not get adopted. These three interlock: the pinned environment makes the README's instructions true, and the test proves both are still true after the author has gone. Keeping the dependency specification honest over time is the part most likely to decay, and dependency management covers how to keep pinned versions from becoming a security liability of their own.
Related Resources
Training and Culture
Making knowledge outlive the people who hold it, and practices that survive a team turning over completely.
ML and Data Debt
Pipeline jungles, data provenance, and reproducibility failures from the industry side of the same problem.
Dependency Management
Keeping pinned environments reproducible without letting them rot into an unpatched security liability.
Research and Citations
The bibliography behind this site, and the sourcing standard every figure on it has to meet.
Frequently Asked Questions
Because the funding and career structures produce it directly rather than incidentally. Grants pay for novel work and rarely for maintenance, so sustainability has no funding mechanism. Positions are fixed-term by design, so the most knowledgeable contributor is guaranteed to leave on a schedule known in advance. Advancement is measured in publications, so time spent on maintainability is time spent on something nobody counts. And the consequence differs: elsewhere debt makes software slow to change, whereas here it can make a published result impossible to verify.
A README that actually works, a pinned environment, and one test that proves the pipeline runs. The README should be instructions a newcomer can follow on a clean machine to reach a correct result, not a description of the project. The pinned environment is an exact version-locked specification committed alongside the code, because without it the code is only half of the experiment. The single test compares the whole pipeline's output on a small fixed input to a stored expected result. They interlock, and longer lists do not get adopted.
Lead with a problem they already have rather than a critique of their code. Nearly every group has a felt pain: an analysis that takes a week to rerun, a result that could not be reproduced, a reviewer asking for a sensitivity analysis that means editing a parameter in fourteen places. Fix that with them rather than for them. Pairing works; rewriting their analysis over a weekend does not, because they can no longer modify it. Use the vocabulary of experimental method rather than software architecture, and remember the expertise gap runs both ways.
A Research Software Engineer combines software engineering skill with enough domain knowledge to work inside a research group. The US-RSE Association puts it simply: Research Software Engineers use expertise in programming to advance research. Institutions are creating central RSE groups because that is the only funding shape matching the demand. An individual grant cannot sustain a maintainer, but an institution can spread the cost across many projects, retain the expertise between grants, and offer a career track with progression in it -- which is what was missing before.
By routing the work into the currency that already counts instead of arguing the currency is wrong. Add a citation file to the repository so users know exactly how to cite the software, turning downstream use into something that appears in metrics. Archive releases to obtain a persistent identifier, which makes a specific version citable and guarantees the version behind a published result still exists. Publish a software paper where a journal accepts them, since a well-used tool can accumulate citations for years after its funding ended.
Two things, predictably. Volunteer work arrives in bursts from people who cannot be asked to come back and finish, producing a codebase of half-migrations: two authentication approaches, three date conventions, a partially completed framework upgrade. Each contribution improved something and collectively they left a system nobody has a mental model of. Donated infrastructure creates architectural commitments with expiry dates attached to somebody else's programme. Scope volunteer contributions so a single session can finish them, and know which pieces of a donated platform are portable before the renewal notice arrives.
Build Software Your Successor Can Actually Run
The incentives do not reward maintenance, so pick the few practices that survive them: a README that works, a pinned environment, one test that proves the pipeline runs.