Successful MMIS Implementations (Part 3 of 3)

So what action the community take to address the challenges and barriers related to MMIS implementation?

Prior blog entries were an attempt to shed light on the day-to-day situation of an MMIS project and should not be read as complaints or issues directed at any one entity. Good project management practices can and should deal with the issues such as vague requirements, relationship management, fiscal and schedule performance, and project communications. However, the probability of success of MMIS projects can be tremendously increased with a change in framework and approach. We should add that the issues discussed in Parts 1 and 2 are not unique to the MMIS market. Software development projects across all industries face similar situations. The general trend in all industries is that the penalty for underestimation is more severe than the penalty for overestimation. The FFP strategy can be successful, but it is a blunt instrument in current practice. The desire is not to let the pendulum swing completely away from FFP contracts back to the time and materials environment. A balanced approach is warranted and can be implemented to meet everyone’s goals and objectives. The following provides specific recommendations for future MMIS success. We have organized these recommendations by the key stakeholders involved in each MMIS contract: State, CMS, and Vendor.
Recommendations for State Agencies

• Balance between time and materials (T&M) and fixed price contracts. Consider breaking the MMIS procurement into at least two phases: phase 1 - requirements and design and phase 2 – development and implementation. The T&M contract vehicle can be used for phase 1 and FFP for phase 2. It should be noted that it is assumed that the same vendor is recommended to complete both the phases as the context of requirements is bound to be lost in transition if a different vendor is brought in for phase 2.
• Implement award fee or other incentive based contracts. The state has certain goals and objectives for the MMIS procurement. The vendor is motivated to perform. However, the two are not necessarily aligned. Is the vendor motivated to addressing the goals and objectives? Award fee contracts used often in the Federal government contracts are an excellent vehicle to align goal achievement and performance and rewards. The Census Bureau and the Federal Bureau of Investigations have recently awarded these types of contracts.
• Consider performance based contracting (PBC). Again, this is a popular approach in Federal government. The details of performance based contracting are not within the scope of this paper; however, in general PBC focuses on performance and service level agreements. It establishes a quality assurance surveillance plan for contracts administration and this plan is tied intimately to the vendor’s quality assurance and project management.
• Establish statements of objectives (SOO) and a concept of operations document in the RFP and avoid specific requirements. Federal agencies such as the Federal Bureau of Investigation, Department of Defense, and Department of Commerce follow this approach when procuring complex information systems. The general idea is to establish objectives and describe the vision of the operations of the MMIS for the vendor community. The vendors then conduct requirements elicitation during DD&I. This approach avoids the loss of the requirements context discussed earlier in this paper. The vendor’s analysts and design staff will be able to capture precise information necessary to build the MMIS. It also avoids frustrations faced by the state staff in having to participate twice in the same process, namely, during requirements gathering by the vendor helping to write the RFP, and then during the requirements analysis/validation by the vendor building the MMIS.
• Allow the vendor to propose an alternative system development life cycle (SDLC). Most MMIS RFP’s require a waterfall-based SDLC. Vendors can help manage risks and be more responsive with alternative processes such as agile and rapid prototyping. Require parallel system testing. The primary purpose of an MMIS is to pay claims and it is absolutely critical to ensure that differences (if any) in payment amount are fully understood between the old MMIS and the new MMIS. Providers are very politically involved and motivated to use all avenues if the new MMIS pays claims differently than the old MMIS; even if the system is paying more accurately. Parallel testing is a great risk mitigation process prior to going live with the MMIS. Some states recommend parallel testing but this process should be mandated and should not be compromised in any circumstance.
• Avoid big bang implementations. Implementing a large system in one deployment is difficult to manage given the large number of stakeholders and thousands of unknown variables. Many states evolve to some form of phased implementation during DD&I; however, it is a very disruptive process to change to a phased implementation after creating and pricing a work plan for a big bang implementation. It is much easier to plan for a phased implementation as a part of the procurement process.
• Train state business users on how to manage FFP contracts and the project management triangle.

Recommendations for CMS

• Entertain and require alternative acquisition strategies (i.e., T&M, award fee, etc). Ask states to reduce risk and consider alternative acquisition strategies as a part of the Advanced Planning Document approval.
• Be more actively involved during the DD&I phase. CMS is a key stakeholder that funds up to 90% of the DD&I phase of the MMIS. CMS involvement as part of the Project Management Office and governance processes is critical for proper communications. The model of current involvement is passive in terms of status reports and occasional site visits. CMS involvement should be more integral to the project execution than otherwise. It keeps all three parties (i.e., CMS, state, and vendor), fully engaged in the success of the project.
• Consider mandating IV&V activities and/or establish a Project Control Office vendor that reports to CMS. Collect history of project performance on past MMIS implementations and share lessons learned across states. As already mentioned above, MMIS implementations vary; however, there are many similar traits. It would be helpful to have a standard repository of project performance information and a forum for sharing of lessons learned.
• Encourage state/group conferences with the vendor. Encourage states to conduct conferences with the common vendor amongst the states. This will allow states that have similar systems to share best practices and lessons learned in implementing the system.
• If possible, coordinate RFP releases so that they are not all coming out at the same time. Having multiple RFPs out at once reduces competition or dilutes bids. One of the ways that CMS can increase competition is by providing grants to vendors for research and development of new systems and functionality.
• Facilitate a User Group organization that is focused on states with similar issues; states that are embarking on implementing new systems; states that are constructing new systems; and states that have implemented systems. Lessons learned shared from one state to the other will be invaluable. Also, states will be able to share common system components that have generic applicability. Formalize this process since there is already some level of Regional Exchange that exists. This formalization gives it meaning and making it a mandatory process as a part of reporting updates to CMS during the project execution.

Recommendations for Vendors

• Train vendor staff to handle FFP contracts. Project managers may have the knowledge or experience; however, team leads and analysts are often required to face state staff without adequate preparation on how to manage scope.
• For a state to make a good decision, the vendors need to be more open and honest about their staffing and pricing models.
• Vendors should train their staff better prior to engaging with a state. Vendors always have an issue where some of the personnel involved do not know the details of the solution being offered.
• Vendors should be more transparent in their estimation process and willing to share that with the state and CMS during the RFP response.
• Vendors should establish a framework for more state staff quality participation during all the phases of the DDI implementation and not limited to requirements and design phases.
• Communicate openly with the state and with CMS. Don’t become too “bogged down” in competitive posturing to the detriment of the project.

Read more!

Successful MMIS Implementations (Part 2 of 3)

This is a continuation from Part 1...

In this part, the barriers to success are discussed...

Managing the Cost, Scope, and Schedule Triangle

MMIS procurements are are generally conducted in a very risk averse environment; however, the projects often desire new technologies and innovative approaches to solve current and anticipated business problems. In addition, business objectives and goals are sometimes overshadowed by deadlines for federal mandates and an expiring current MMIS contract. A fixed price contract is a convenient mechanism for states to shift risks to the vendor; costs are capped, and scope is in theory contained. On the surface it is an attractive situation -- the project management triangle is established where cost, schedule, and scope have been defined. However, if scope or schedule change, there is an obvious impact on the cost. Having a fixed price framework does not allow the cost to change hence compromising scope and schedule.

Loss of Context during the Request for Proposal Development Process

State agencies have to start the MMIS procurement Request for Proposal (RFP) very early in order for it to go through the acquisition process. The term used for procurement documents varies from state-to-state but the documents are collectively referred to as the RFP in this paper. As mentioned at earlier, the high cost, visibility, and political nature of these projects generally means that many stakeholders have to be involved and it naturally takes time to address everyone’s needs. State agencies typically undergo a long and arduous process to collect the RFP requirements from their stakeholders. These requirements are usually gathered through either formal or informal requirem ents elicitation sessions. Meaning, anticipated users of the MMIS and other stakeholders involved in MMIS business processes are solicited to capture the requirements for the new system. The exact process and the degree of formality vary from state-to-state; many states contract with consulting organizations that have expertise with Medicaid to help develop the requirements for the RFP (and perhaps develop the entire RFP). A significant negative side effect of this approach is that much of the valuable information related to the requirements is usually lost in the process. The context surrounding each requirement is very important to the vendor responsible for DD&I. The context includes information such as who asked for the requirement, what was the intent behind the requirement, relative priority, discussions, diagrams, and other artifacts. The value of requirements elicitation is often not in the exact requirement statement resulting from the process but rather in the discussion around the requirement. Good analysts probe and ask questions and tackle a specific need from a variety of angles as they form the actual requirements statement. However, the information conveyed during that discussion is difficult to capture in an RFP context. The English language is also not precise enough to convey the information since there are many ways to interpret statements. Also, requirements are not prioritized; a particular requirement may be high-effort and low return.

The time period between when state stakeholders first meet to identify the requirements to the point when the DD&I vendor conducts a requirements verification or validation session can range from one to two years. By that time, it is very difficult to remember the intent behind the requirement. Stakeholders sometimes question why a requirement was even in the RFP during a DD&I requirements verification/validation session. This is often the case because requirements gathering exercise take place at too high a level in the organization. Precious DD&I time has to be used to uncover history and to attempt to bring back the context of the discussion around a requirement. That time period is difficult to predict and quantify while developing a proposal response to the RFP. The consultant or group that conducted the RFP requirements elicitation for the state is often not available once the contract is awarded to the vendor.

Requirements Maturity Challenges Scope and Estimation Processes

Large, complex systems are difficult to define and specify. The procurement documents (i.e., Request for Proposals) define requirements for these systems; however, the requirements are often broad and high level and, as mentioned above, much of the context around each requirement is lost in the process. The requirements for the MMIS are represented as one or two sentence statements. They are often organized around the functional areas of an MMIS such as Claims, Provider, and Member. Over the past several years the RFP requirements have become more detailed and thorough. The sheer number of requirements in RFPs for the entire MMIS has grown from around 100 to over 2,000 over the past four years. However, the requirements are still very broad and high level. Consider these examples from recent RFPs:

• “Ability to apply Expedited Prior Authorization criteria to claim data”
• “Ability to query on provider attributes, as defined by the user”
• “The system must support rebate negotiations”
• “[The system must provide] Enhanced premium assistance functionality”

In addition, many of the requirements include phrases such as “including, but not limited to” and “etc.” that make it extremely difficult for the vendors to bind the scope. RFP requirements are also often not written to follow industry best practices to be clear, concise, and testable. Vendors are required to estimate a two to three year DD&I period based on requirements written to the specificity above. It is a difficult task. So the FFP price will often reflect the risk of the unknown. To bind the scope, vendors will often document assumptions as a part of the proposal in response to these requirements. These assumptions may prove to be inaccurate during DD&I resulting in some form of a conflict between vendor and the state agency. The question and answer period during the procurement process is not adequate to explore and probe vaguely written requirements because there is limited opportunity to have face-to-face contact with the end users and, as discussed below, the right resources and thought process behind these requirements is lost. The end result is that the scope side of the project management triangle is not as solid or firm as is assumed when undertaking the FFP contract for an MMIS project. At best, the schedule side of the triangle is murky given the vagueness of the requirements. Scope and schedule uncertainty eventually impacts the cost.

So what can we all do to address these challenges? Stay tuned... Read more!