Domain 1- System Infrastructure- This domain examines the technology the application is built upon. This includes everything from the operating systems at the user level, the server operating systems, server software for web servers, applications server architecture, database servers, development tools, deployment tools, and any technology required directly or peripherally to the software. The ability for local staff to support and maintain and generally work with the software components is also important as well as the ability to find human resources locally who are familiar with the components. The ability for local staff to work with and support the technology is also an important consideration.
- Sub-domain 1.1 Technology Supported (architecture, programming language, database) Software systems are created using different tools, languages, databases and they can be structured very differently. These components change over time with some components becoming less popular as newer tools become available which may offer advantages over previous offerings such as ease of use, execution efficiency, speed, and reliability among other things. New tools and components are always coming on the market and some are less mature than others; the new tools may offer advantages but may not have the long-term in-field experience and testing.
- Sub-domain 1.2 Availability of Local expertise to support and maintain the software. In some cases the purchaser will be allowed to make modifications to the software and local developers will need to know how to utilize the entire infrastructure to make those changes. In other cases, the software provided is complete but the environment the tool operates within will need to be managed by local personnel and consideration needs to be taken that local expertise exists to manage the operation systems, databases, communications protocols, software servers and other components. Some systems are more commonly utilized in different regions and there are more people available who can perform the work. Some systems utilize tools and components which are more obscure and there are fewer people who understand them.
- Sub-domain 1.3 Software Scalability This refers to the capacity of the tool to handle more users, transactions, requests and storage at any given time and over time. Some systems will not need a high level of scalability but other systems will need to handle high volumes of traffic and data. It's good to evaluate a system to determine if it's possible to increase the capacity and through-put and how easy it is to increase how the system handles increased volume in various aspects.
- Sub-domain 1.4 Installation Scalability This refers to the level of effort required to distribute and install the tool in new installations.
- Sub-domain 1.5 Third-party licensing of components required Some tools will have costs in addition to the cost and licensing of the tool itself. Databases sometimes require licensing fees from other companies but the tools used to develop the application may also be specialized and the purchases may be required to license those components if you intend to make programming level changes. Additional licensing is common is for reporting software to create analytical reports. Hosting can sometimes fall under this category if the tool is required to be hosted by a third-party.
- Sub-domain 1.6 Technology and internet/connectivity Evaluate the general use of the tool and it's need to interact with the rest of the world. Tools have varying degrees of requirements for inter-communication or access to/from systems from the outside world. The system may be a stand-alone system that runs on a single computer or it may be a fully connected system which require a high-speed internet connection any transactions to take place. There are many different degrees and methods of the need for networking and connectivity.
Domain 2- Utilization- This domain examines the breadth of the user base and utilization in the target countries. Some tools are utilized heavily in a low number of settings and other might many countries utilizing the tool but one installation per country. Some tools may only need small number of installations in each country if the tool is used at a central level by the Ministry of Health. Another tool might have dozens of installations in each country.
- Sub-domain 2.1 Number of countries and total installations in each country This is a difficult number to quantify. Examine the link between the number of countries which utilize the software and how many installations there are within each country. Also consider how the tool is utilized and how important each factor is based on the tool; some tools may only require one installation per country while others may need multiple installations or an installation at each location it is to be utilized.
- Sub-domain 2.2 Number of countries Installations in each country
- Sub-domain 2.3 Installations in each country Each installation may rely on and utilize the tool in very different ways. For existing customer this Sub Domain refers to how heavily existing installations rely on the tool and whether the tool is critical to their day to day work. One other way to think about this is what level of disruption to workflow would be caused if the tool is unable to be utilized? Some sites would likely easily be able to continue work and some sites would have a significant work slowdown or stoppage if the tool could not be utilized.
- Sub-domain 2.4 Tool up-time
- Sub-domain 2.5 How fully is the software utilized (10% of functionality or 100%)Some tools have a lot of functionality and different modules which may or may not be utilized at each location. It's not bad that locations don’t utilize every aspect of the tool. Locations may not utilize most of the functionality but they rely heavily on the functionality they do use. Some tools have pieces of functionality which is heavily utilized by almost all locations but most of the modules may not be utilized at all either because they are not as expansive as the location needs or the location simply does not need the functionality.
- Sub-domain 2.6 How long has the tool been in use? It's not simply a matter of when the software was introduced but has the number of locations utilizing the software changed over time and also the reliance on the software over time. Some tools may not have even been possible to have been created more than a few years ago. Other tools might have been in use a long time but are struggling to keep the existing installations or have new locations implementing the tool.
Domain 3- Country eHealth Strategy
- Sub-domain 3.1 The level of recognition and official government support given to a tool often depends on how well it is integrated into the official policy of the ministry of health. This may not be true of all countries but official recognition and policies surrounding a tool or software package often give the tool more credence and support from the government and help towards the long term sustainability of the tool.
- Sub-domain 3.2 The level of inclusion of the tool builders have with the laboratories and staff using the tools can have a significant impact on the useability and functionality in a tool. Engaging in user groups and communicating requests with the tool supplier is always encouraged.
- Sub-domain 3.3 Georgraphic coverage in a country
- Sub-domain 3.4 User centred design
- Sub-domain 3.5 Supporting lab functionality
- Sub-domain 3.6 Governance structure
Domain 4- Roadmap- Provides a detail understanding of intent and direction in which the company /tool provider plans to go, serving as a strategic communication tool. Due to evolving technologies and user expectations, determining future needs is challenging. Tool vendors/providers are having to change their approach to developing roadmaps and adopt different procedures for the development and management of product roadmaps.
- Sub-domain 4.1 Knowledge sharing
- Sub-domain 4.2 Roadmap detailing Detailing of roadmap with respect to timeline
- Sub-domain 4.3 Roadmap Items Items included in tool roadmap
- Sub-domain 4.4 Reliability of Roadmap Frequency of roadmap adjustment
- Sub-domain 4.5 Confidence in Roadmap Confidence in roadmap items having expected impact on goals
- Sub-domain 4.6 Product discovery Conducting product discovery
- Sub-domain 4.7 Prioritization process
- Sub-domain 4.8 Extent of Alignment of stakeholders
- Sub-domain 4.9 Responsibility for placing items on roadmap
- Sub-domain 4.10 Ownership Who owns the tool roadmap and is accountable
- Sub-domain 4.11 Commercial product development plan
Domain 5-Technical Support and Multi-Language- The day to day users who interact with the tool will have questions and encounter problems. There are different models to support those users. One convenient method is to have a small number of staff who receive extra training and become "super users". The regular staff should have easy access to these super users to ask questions and resolve issues. This is a convenient model because the super users are typically staff at the same location or nearby, speak the same language and are familiar with the specifics of the installation. Another method for supporting the end users is a help-line they can call which is staffed by the tool provider or other the implementing organization. For this model consideration should be made for the languages spoken, the ease with which someone can be contacted, how easy they are to communicate with and any language issues. Some tools are easier than others to support remotely. Other considerations are the ability to support the tool by remote login or even the availability and ease/cost of on-site visits for troubleshooting more advanced issues. It is also more common to have online forums or knowledgebases that can be searched for information about the tool.
- Sub-domain 5.1-The day to day users will encounter problems and have questions. There are different models which can be implemented to make the ease with which users can ask questions and get problems resolved.
- Sub-domain 5.2-There are more and more online services to allow users or super users to get questions answered or generally find out more information. These tools could online submission of questions and problem reports directly with the tool provider (electronic help desk), online chat sessions or other direct messaging services like text or WhatsApp, etc. There are also more and more online user forums and knowledge bases which supply information about the tools.
- Sub-domain 5.3-Users with more advanced training may be able to skip the first-line end-user support queue and go directly to a more advanced support service. Or they may be required to go through the regular service. The same considerations apply in regards to the ease and speed with which advanced users can get more advanced support. The issues a super user needs assistance with may be more critical to the operation of the system which makes their access to support an important issue to consider.
- Sub-domain 5.4- Some tool providers may have a first-line phone support service which is located close to or within the same country or they may only have a single line which goes directly to the primary support line.
- Sub-domain 5.5- During the implementation phase of a tool it is common but not always required for onsite visits by the tool provider for requirements gathering, training and validation. Sometimes it is necessary for support staff to visit an installation and trouble-shoot a problem. The ease with which someone can visit a location is a consideration when thinking about how much disruption would be caused if the tool cannot be utilized. With the availability of remote computer access this has become less of an issue but for more advanced investigations it can sometimes be needed. Some tool providers may have local companies they partner with who do installations and support and while they may not have the advanced knowledge of the tool provider's staff they are often information systems experts and have very advanced training on the tool and related systems.
Domain 6- Technical/Developer Documentation- The degree to which a tool can be modified varies greatly. Some tools are completed locked and there are relatively few configurations changes that can be made. Some tools may have their source code locked but have an extremely high degree of reconfigurability on top of the base product including the ability to add new storage tables and user interface changes. Some tools will allow complete access to the underlying source code of the system. The ability to make highly technical changes to the system needs to have some level of documentation. In the case of configurability this documentation is extremely valuable. Where source code is accessible the documentation should focus less on the specifics of the underlying tools and languages and focus more on how to set up the development environment, specific versions of tools and libraries, descriptions of where specific types of changes should be made, and how to create the final product utilizing all those tools. If the software is part of a shared codebase like an Opensource software system, the documentation should also be very specific about how to interact with that shared code repository.
- Sub-domain 6.1 The percentages aren't meant to be precise but to give a general expectation about how much of the overall development of the tool is documented. Regardless of whether the changes through configuration changes or through programming language changes what amount of this is clearly documented will give you an idea of how easy the changes will be to make.
- Sub-domain 6.2 As with user support forums and knowledge bases have become invaluable tools for developers and highly technical staff to utilize to ask questions and to find solutions to problems or needs that other people have already asked and found appropriate responses.
- Sub-domain 6.3 The level of support documentation provides to programmers/technical staff for understanding tasks
- Sub-domain 6.4 The lower the granularity the closer the documentation is to the implementation of the tool, and the higher the granularity the close the documentation is to the business processes of the tool
- Sub-domain 6.5 Technical documentation processes
- Sub-domain 6.6 Documentation needs include local language support, screenshots, detailed steps to perform all common functions end users perform. Documentation of needs or procedures specific to a particular location is not included in this evaluation
- Sub-domain 6.7 Super user documentation includes details steps of all system configuration options and procedures, system architecture, regular maintenance procedures, troubleshooting guide for common issues. Documentation should be available via the internet but also locally in case there is no internet connectivity.
- Sub-domain 6.8 Knowledge base or user forum for user and super user level questions
Domain 7- Tools vary greatly in the level of effort required to get the tool installed and integrated into the workflow of the laboratory. Some tools will be able to be installed remotely by the provider or require very little site-specific configuration or training. Some systems may be so critical to the laboratory workflow they required detailed requirements gathering and gap analysis to determine small changes to make sure they conform to the local laboratory's workflow as well as also take time and expertise to make sure all of those requirements are captured. In general, the more dependent the staff are on the tool as part of their workflow you should expect a similar level of configuration and time required for implementation.
Sub-domain 7.1