This article is reproduced, with permission, from the website of OSS-Watch, a project providing information and guidance on open source software for FE and HE. Its messages are equally applicable to schools.

Performance and reliability are the principal criteria for selecting software. In most procurement exercises however, price is also a determining factor when comparing quotes from multiple vendors. Price comparisons do have a role, but usually not in terms of a simple comparison of purchase prices. Rather, price issues tend to arise when comparing total cost of ownership (TCO), which includes both the purchase price and ongoing costs for support (and licence renewal) over the real life span of the product.

Some frameworks have been developed to help those in IT procurement assess open source software. These can help determine the appropriateness of particular applications in specific situations, or to evaluate different open source products against one another. They are not so well suited for comparing open source software against proprietary alternatives. Examples of such frameworks include the Business Readiness Rating (BRR) and the Open Source Maturity Model (OSMM).

Reputation

Does the software have a good reputation for performance and reliability? Here, word of mouth reports from people whose opinion you trust is often key. Some open source software has a very good reputation in the industry, e.g. Apache web server, GNU Compiler Collection (GCC), Linux, Samba etc. You should be comparing best of breed open source software against its proprietary peers. Discussing your plans with someone with experience of using open source software and an awareness of the packages you are proposing to use is vital.

Ongoing effort

Is there clear evidence of ongoing effort to develop the open source software you are considering? Has there been recent work to fix bugs and meet user needs? Active projects usually have regularly updated web pages and busy development email lists. They usually encourage the participation of those who use the software in its further development. If everything is quiet on the development front, it might be that work has been suspended or even stopped.

Standards and interoperability

Choose software which implements open standards. Interoperability with other software is an important way of getting more from your investment. Good software does not reinvent the wheel, or force you to learn new languages or complex data formats.

Support (Community)

Does the project have an active support community ready to answer your questions concerning deployment? Look at the project's mailing list archive, if available. If you post a message to the list and receive a reasonably prompt and helpful reply, this may be a sign that there is an active community of users out there ready to help. Good practice suggests that if you wish to avail yourself of such support, you should also be willing to provide support for other members of the community when you are able.

Support (Commercial)

Third party commercial support is available from a diversity of companies, ranging from large corporations such as IBM and Sun Microsystems, to specialist open source organizations such as Red Hat and MySQL, to local firms and independent contractors. Commercial support is most commonly available for more widely used products or from specialist companies who will support any product within their particular specialism.

Version

When was the last stable version of the software released? Virtually no software, proprietary or open source, is completely bug free. If there is an active development community, newly discovered bugs will be fixed and patches to the software or a new version will be released. For enterprise use, you need the most recent stable release of the software, where there have been many recent releases in the unstable branch of development. There is, of course, always the option of fixing bugs yourself, since the source code of the software will be available to you. But that rather depends on your (or your team's) skill set and time commitments.

Version 1.0.

Open source projects usually follow the release early and often motto. While in development they may have very low version numbers. Typically a product needs to reach its 1.0 release prior to being considered for enterprise use. This is not to say that many pre-1.0 versions of software are not very good indeed, e.g. Mozilla's 0.8 release of its Firefox browser was polished and mature.

Documentation

Open source software projects may lag behind in their documentation for end users, but they are typically very good with their development documentation. You should be able to trace a clear history of bug fixes, feature changes, etc. This may provide the best insight into whether the product, at its current point in development, is fit for your purposes.

Skill set

Consider the skill set of yourself and your colleagues. Do you have the appropriate skills to deploy and maintain this software? If not, will you employ third party contractors or will you implement a training plan to match your skills to the task? Remember, this is not simply true for open source software, but also for proprietary software. These training costs should be included when comparing TCOs for different products.

Project Development Model

Open source development should not be chaotic, although it can sometimes look that way. An open source project should have a very clear development process that describes how contributions are made and how they are evaluated for inclusion. It should also describe how contributors investing considerable resource in customisations can become a part of, or influence to, the project management. This is to reassure significant contributors that their contributions will remain available and valuable to them in the future. In some projects there is a formal structure governing this kind of development, in others the structure is fluid, in both cases the rules of engagement need to be clear.

Licence

Arguably, open source software is as much about the licence as it is about the development methodology. Read the licence. Recognised open source licences have well defined conditions for your contribution of code to the ongoing development of the software or the incorporation of the code into other packages. If you are not familiar with these licences or with the one used by the software you are considering, take the time to clarify conditions of use.

Coverage

Many open source products are generalist and must be specialized before use. Generally speaking the more effort required to specialize a product, the greater is its generality. A more narrowly focused product will reduce the effort required to deploy it, but may lack flexibility. An example of the former is GNU Compiler Collection (GCC), and an example of the latter might be the Evolution mail client, which works well out of the box but is only suitable for the narrow range of tasks for which it was intended.

Note: This document is derived from the document www.oss-watch.ac.uk/resources/tips.xml published by www.oss-watch.ac.uk. The latest quality assured OSS-Watch version is always available on the OSS-Watch site. Unless otherwise indicated, this page is © 2008 University of Oxford. It is licensed under the Creative Commons Attribution-ShareAlike 2.0 England & Wales licence.