Wednesday, September 19, 2007
Version Control Systems as Research Tools
For a while, I've been contemplating the use of source code version control systems (such as CVS) as a tool for basic computer science research. This setup supposes that a system will be running some experiments on a set of input data, and that after running all experiments, some text-based report will be generated. The setup further supposes that structural or algorithmic changes (as opposed to parameter changes) will be applied to the software between experimental runs. By keeping the source code and result report under version control, a record of all results will be kept. Additionally, past experiment configurations can be recovered by simply accessing previous versions of the software in the repository. Moreover, since CVS only stores file changes (deltas), the effects of various changes will be immediately apparent; existing GUI or web based tools will literally highlight how each result report changed with each software modification. This could really help/formalize future analysis, and may help justify final results.