Let’s rewrite scientific tools manually
Software engineers are commonly told not to reinvent the wheel. You are unlikely to make a better hash table, or web framework, or database engine, than what others have already done. It’s often, and with good reason, considered best practice to build on established components that are already in widespread use. But should we sometimes rewrite software? What could we stand to gain from it?
I think the warning against reinventing the wheel applies best to junior developers who have not yet understood the limits of their own knowledge, and who might not have a sense of how much work and experience has gone into many of the components we take for granted every day. In that situation it’s easy to naively take on a task of massive scale and create a lot of technical debt in the process (having been one of those juniors myself once). But the only way to get a sense of what such a task demands is sometimes to attempt it yourself.
In scientific software, as in many other fields, we routinely trust that implementations of methods do what they say they do. We trust artifacts. We take the authors at their word. While it is not necessarily bad to do so, good scientific practice should also involve reproducing other peoples’ methods and results. Tools, theories, and equipment need to be as transparent as possible for the scientific process to be sound. Perhaps we, in so far as we wear our scientist hats, ultimately care more about understanding itself than about some idea of productivity.
When somebody does successfully reimplement a scientific tool, perhaps in a different programming language or software framework, or with a different architecture, something else happens, too. Suddenly we can distinguish between a method and its implementations. In biological terms, we have turned an individual into a species. If an algorithm previously had only one practically usable implementation, then trusting the method would have been the same as trusting the only implementation with all its idiosyncrasies. There would have been no difference. But with multiple implementations in existence, they can be compared against each other and such quirks can be discovered. Suddenly the differences between specific implementations and the ideal form of the method becomes apparent.
As part of our work, we have rewritten (and extended) the metagenomic classifier Kraken 2 (in Slacken) and the k-mer counter KMC3 (in Discount) using a different architecture and different programming language (distributed computing, using Apache Spark, with the language Scala), and doing so taught us a huge amount about both the target architecture, the “source architecture”, and the subtle details of the original implementations.
For users who can take the time to attempt a rewrite of a scientific tool it is almost always a good idea to do it, even if they end up failing to complete the project and having to throw their work away. At worst, in the process they will be able to form a deep understanding of the original tool and perhaps uncover new insights about it. At best, they will have a better implementation and/or improvements to contribute to upstream. I would also encourage people to avoid using AI for doing such rewrites. With AI, a result can perhaps be obtained much faster but the programmer hasn’t learned very much. As a researcher, would you trust a scientific tool that was stochastically generated? Would you draw conclusions based on its outputs?
Seqera, the company behind Nextflow, recently launched a fascinating project called rewrites.bio, a manifesto for the use of AI to rewrite tools in Rust. The manifesto seems to have many good ideas (and rewriting tools in Rust is in itself a good idea), but one of the open questions for me would be how we validate AI-rewritten tools. Proving that two tools produce the same results is in general a hard problem. It would be better to do such rewrites manually if possible, and in any case there would need to be a trusted process for validation when this is done at scale. If we could establish such a validation process, it would benefit both manual “rewriters” and AI rewriters.
Manual rewrites can also have personal value for the individuals who do them. Humans who can read and understand existing code are in short supply, and the need for this hard-won skill will not go away - if anything, we may soon need more of it. Scientific engineers who spend time learning to do this are not only investing in their scientific field, but also investing in their own careers, and in our technological future. Only humans who can read and understand code can be trusted to write or validate code that must be correct (a fact that many people are trying to gloss over in the present business climate). Robert Martin, author of Clean Code, made similar comments in a recent interview.
As people increasingly lean on AI to write code, programmers rarely have a chance to face the blank page, to create a project from scratch. This is a muscle that AI will take from young engineers as AI provides a quick path to the first prototype. I would encourage people to still do a project from scratch at least a couple of times in their lives to build this muscle. Why not rewrite a scientific tool? This would be an investment both in a scientific field and in individual skill, knowledge and understanding, both of the science and of the software engineering.