A strategic solution for purchasing organizations

A group decision support system for RFP evaluation

A strategic solution for purchasing organizations

Complete eRFP Tutorial Available at:

http://erfp.sysoft.com

100 Wood Ave SouthSuite 111Iselin, NJ 08830

Ph: [email protected]://erfp.sysoft.com

Bringing RFP Evaluations Into The 21st Century

Paul J. Brennan, CPPB, C.P.M.

For years, those in the procurement field, both public and private, have heard the expectations of how our profession would dramatically change from the implementation of e-procurement technology. While most of those in the procurement profession will certainly agree that our jobs today have changed dramatically over the past five years; it remains a fact that many of the processes involved in source selection have seen little change or have remained basically the same.

While most of the Enterprise Resource Planning Software and Procurement Software that is available today has embraced “e-procurement” by offering E-Bidding, Bid Tabulation, Automatic Supplier Notification, and other new features, most if not all of the software providers have not provided an automated solution to the evaluation of Request for Proposals.

As more and more government organizations move away from the “low bid or low cost approach” to contracting, the Request for Proposal process is becoming the preferred method of source selection to identify the “best qualified” or “best value” proposal.

Rockland County, New York, like most other government agencies had seen their use of the Request for Proposal increase significantly over the past five years. Already issuing over 225 Competitive Bids on an annual basis, Rockland began to see the number of Requests for Proposals issued each year reach an average of 50 – 60. Along with the increased use of the RFP came many operational issues including, how to manage multiple evaluations being conducted at the same time, how to ensure thorough, independent and objective evaluations by multi-disciplinary teams, how to take evaluation data from paper rating forms and roll that information up to meaningful reports that can assist the evaluation teams in making informed decisions, and how to document the decision making process so that the source selection decision can be justified to administrators and elected officials.

Rockland County determined that it needed an automated group decision support system to help manage the RFP evaluation process. With no knowledge of such software currently available, the County issued a Request for Information, which was advertised nationally to attempt to locate such a software product. The County received many responses with offers to build such a system from scratch; and only two responses from software providers that had similar software available or in development.

One of the responses came from a software provider named Sysoft for a product called eRFP. eRFP was designed as a group decision support system for RFP evaluations. At the time of Sysoft’s response to the County’s Request for Information, the design of the eRFP software was ninety-five percent complete but had yet to be beta tested. The timing of our RFI could not have been better as the County identified a new software product as well as a software provider that was looking for a government agency to partner with to help beta test the software and identify any design changes needed based on using the system to evaluate actual government projects. The result was a contract that benefited both the County as well as Sysoft.

Since implementing the eRFP evaluation system the County has come to rely on the use of this on-line evaluation tool to make better sourcing decisions. The system is database driven and uses the most current web-based technologies. The software is deployable through an organization’s Intranet or through the Internet. Internet access allows evaluators in many different locations to easily evaluate the same proposals. Since the software is web-based, it also allows evaluators access from their home computers. Allowing evaluators to work from home while evaluating proposals can lead to more in-depth evaluations, usually in a shorter timeframe.

Rockland County’s experience has been very positive from the beginning of this project. There have been the normal bumps in the road or “bugs” that are normally experienced when one is participating in a software development project. The use of a web-based software product has enabled Sysoft to respond quickly to any software issues that arose during this project.

From a procurement management standpoint, the eRFP software has permitted us to track the progress of each evaluator, capture evaluator’s comments for use at caucus meetings, identify strengths and weaknesses of each proposal, score proposals in a more objective and less subjective manner, and produce detailed reports that assist in the decision making process.

Using the eRFP software has given the County the ability to easily identify the “best qualified” firm or proposal. By ensuring the selection the most qualified firms, the County anticipates that it will realize cost savings by having projects completed on time and within budget.

The eRFP software has highlighted the various effects of using “Cost” as an evaluation category and how the use of cost as an evaluation category may skew evaluation results; resulting in lesser qualified firms or proposals being selected. This software has also enabled the County to begin including “Risk Assessments” as part of the evaluation process. Proposals are now evaluated in relation to Risks associated with the proposed approach or Statement of Work; Risks associated with the proposed schedule, Risks associated with the proposed personnel; and Risks associated with the financial capability of the firm to complete the project.

Using the software has also helped the County assess the strengths and weaknesses of evaluators, and has helped identify evaluators who require additional training as well as those who may need to be removed from future consideration as an evaluator.

From the evaluator’s point of view the eRFP software gives them the ability to enter their evaluation on-line. The software guides each evaluator through an organized set of on-line forms with pre-defined evaluation criteria and standards. Evaluators have access to the relevant sections of the RFP that was issued for the project without having to revert to a paper copy of the RFP. Evaluators can record strengths, weaknesses, deficiencies, and statements regarding their rating of each evaluation criteria.

The use of this software has helped provide a more objective evaluation process by having the evaluators complete the evaluation with no knowledge of the scoring formulas, and by asking for comments in relation to each rating they assign. When all of the evaluators assigned to a project have completed their evaluations, rating scores and comments are automatically consolidated into a detailed summary report that is used during a caucus meeting to develop a short list of the best proposals for further consideration. The short listed firms are normally invited to make formal oral presentations. After each oral presentation is completed, evaluators log back onto the eRFP system to score and provide comments on the oral presentation. At this time the evaluators also complete the Risk Assessment of each of the short-listed proposals.

The final evaluation report combines the scores of the written proposal evaluation, the oral presentation evaluation, and the risk assessment evaluation.The software documents the entire evaluation process, from project set-up and the selection of evaluation criteria, to the final award recommendation. The software captures all notes generated during the evaluation process. This documentation helps justify the selection of the most qualified firm or proposal, especially when the cost is higher than other proposals. In summary, the use of software to facilitate the RFP evaluation process has enabled Rockland County to reduce the time and cost of proposal evaluations, reduce the subjectiveness of evaluations, provide a complete audit trail, eliminate paper intensive processes, and increase evaluator efficiency.

Sysoft eRFP proposal evaluation system automates Request for Proposal evaluations. Although a seemingly straightforward management responsibility, the task of managing the evaluation process for request for proposals requires careful administration and the tracking of a substantial amount of information. If the evaluation for request for proposals is mainly a paper driven process, it makes the collection of evaluation data and the development of reports a very time consuming task. Sysoft eRFP automates the source selection process for various types of contracts including but not limited to: Service Contracts, Rental and Lease Contracts, Professional Service Contracts and Software and Technology Contracts. The software assists in providing quality evaluations, complete and traceable documents and timely contract award decisions.

Database driven and Web based Deployable within Internet, Intranet and Extranet The ability for evaluators to enter their evaluation online The ability to view progress and change task assignments of evaluators The ability to assign risk assessments Provides on-line access to relevant acquisition documents Provides electronic evaluation forms with pre-loaded evaluation criteria and standards Guides the evaluator through an organized set of on-line forms to create clarifications,

deficiencies, ratings, rationale and impact statements Ability to store proposal strengths and weaknesses in a notepad area On-line access to all evaluations allows source selection officials to monitor progress and provide

feedback Ratings are automatically consolidated During caucus sessions, individual ratings and evaluations can be viewed on a projector screen Consensus ratings are generated on-line Standard and customized reports can be printed throughout the process Easy evaluator assignment Email notification for evaluation deadlines and automated email/Fax capability

Reduce the time and cost of proposal evaluations Helps reduce subjectiveness of evaluations Availability of complete audit trail Eliminates paper intensive processes Increases evaluator efficiency Instant on-line monitoring of the evaluation process Facilitates more precise evaluations Ensures complete traceability of evaluations Availability of real-time interim and final reports Availability of various cost and other analytical models

Features

Benefits

Experience of Rockland County - New York

ManagementThe most significant benefit of using eRFP is that it provides the ability to easily identify the “best qualified” firm or proposal. By selecting the most qualified firms or proposals, agencies should realize cost savings by having projects completed on time and within budget.

eRFP helps show the various effects of using “Cost” as an evaluation category and helps an agency to determine if cost should be a weighted evaluation category. The system shows how the use of cost as an evaluation category can skew evaluation results; resulting in lesser qualified firms or proposals being selected.

Management gains the ability to track the progress of the evaluators; which helps meet target dates to complete the evaluations. In general the use of the system assists an organization in keeping the evaluation process on schedule.

The system can also help management assess the strengths and weaknesses of the evaluators, and helps identify evaluators who require additional training, and those who should be removed from evaluation teams.

Better EvaluationsThe eRFP system provides a more objective evaluation process by having the evaluators complete the evaluation with no knowledge of the scoring formulas, by asking for comments in relation to each evaluation criteria, and by providing detailed summary reports for use at evaluation team meetings.

The system allows each evaluator to easily score and enter comments for each proposal.

Since the system is web-based, it allows access from multiple locations, including giving evaluators access from their home computers. Allowing evaluators to work from home while evaluating proposals can lead to more in-depth evaluations; usually in a shorter timeframe.

DocumentationThe eRFP system documents the entire evaluation process, from project set-up and the selection of evaluation criteria, to the final award recommendation. It gives the evaluation team the ability to enter notes regarding a specific proposal, or the evaluation process in general.

This documentation helps justify the selection of the most qualified firm or proposal, especially when the cost is higher than other proposals.

This documentation also easily identifies the area in which a proposal is weak, and helps eliminate proposals that may be very weak in one key area.

Process CustomizationThe eRFP system allows customization of the RFP evaluation processes. A library of templates (managed and maintained by the eRFP configurator) makes the eRFP system flexible to handle any RFP process. It also allows evaluation of any type of RFP.

eRFP Process

Step 1

eRFP Process Administrator sets up the RFP evaluation project based on evaluation criteria, customized weight assignments, upset levels, analytical models available in the system, selects evaluators and assigns tasks to evaluators.

Step 2eRFP Process Administrator adds proposals to the RFP evaluation project. This will allow the evaluators assign values to proposals and add detailed comments as they evaluate proposals and criteria assigned to them.

Step 3eRFP Process Administrator adds multiple references. This will allow evaluators or the administrator to assign points based on reference feedbacks

Step 4

Evaluators login and find the proposals waiting for their evaluation. They evaluate, assign ratings to criteria and add unlimited detailed comments for clarification. System monitors and facilitates their progress. Evaluators can generate reports on their own evaluations

Step 5

The process administrator can monitor the overall progress and change task assignments if needed. On completion or at any point of time the process administrator can generate evaluation reports, caucus facilitation reports, add caucus notes, shortlist proposals, award the selected proposal and archive the project for future use.

Cost Models:1. Competitive Range Analysis2. Least Cost Evaluation 3. Greatest Benefit for Dollar Cost4. Cost as an Evaluation Category

Evaluation Criteria:

- Initial written proposals - Technical - Personnel - Relevant Past Performance - DBE/MBE criteria - Cost

- Oral evaluations - Personnel -Team issues - Approach - Technical response - Past Performance

- Risk criteria

Reports: Written Proposal Evaluation

Report Print version of Written Proposal

Evaluation Report Oral and Risk Summary Report Print version of Oral and Risk

Summary Report Overall Combined Report Print version of Overall Combined

Report

eRFP Process Administrator




Evaluators

eRFP Architecture

Access, Oracle or SQL Server

Intranet/ Extranet Cloud

Std. PCs with Windows and Browser


Administration, Data Input, Query and Reports

Database

Web Server


Evaluator

Evaluator

Firewall

Internet Cloud


Evaluator

Intranet/Extranet

(If connection to Internet is required)

Authentication

eRFP – Evaluator Task Assignment

You can assign tasks to evaluators and change the task assignment, if required at anytime.

eRFP – View Progress

You can view progress of the RFP evaluation process at anytime to ensure completion before the caucus meeting.

County of USA Department of General Services - Purchasing Division

eRFP Evaluation System

Written Proposal Evaluation ReportDate: 1/8/2004 Time: 3:26:17 PM

Project Name: RFP-CR-003 Technical Assistance Project Project Number: 0000000144Upset Level Imposed: Yes Selected Cost Model: Competitive Range Analysis

Proposal: EAK Inc. Number of evaluators who completed initial evaluations: 5 out of 5 assigned

Proposal: ATI Number of evaluators who completed initial evaluations: 5 out of 5 assigned

Proposal: KRL and Associates Number of evaluators who completed initial evaluations: 5 out of 5 assigned

Proposal: MultipleSystems Inc. Number of evaluators who completed initial evaluations: 5 out of 5 assigned

Proposal: Reliable Systems Inc. Number of evaluators who completed initial evaluations: 5 out of 5 assigned

Initial Written Proposal Evaluation Ranked by Overall Score

Proposal: MultipleSystems Inc.

Overall Score=3.35

Proposal: ATI

Overall Score=3.32

Proposal: Reliable Systems Inc.

Overall Score=2.72

Proposal: EAK Inc.

Overall Score=2.24

Proposal: KRL and Associates

Overall Score=2.21

Overall Caucus Comments:

Proposal: MultipleSystems Inc.Shortlisted Proposal Initial Evaluation of Written Proposal

Yes MultipleSystems Inc.

Technical Criteria

Weight=40%

Weighted Score:

Personnel Criteria

Weight=30%

Weighted Score:

Past Performance

Weight=30%

Weighted Score:

DBE/MBE Criteria

Weight=0%

Weighted Score:

Cost Criteria

Competitive Range Analysis

Caucus Comments:

Technical Criteria Criteria: The Proposal showed a full and complete understanding of the need(s) or

problem the proposed solution will address

Evaluator: Sam Adams Comment: Solution anticipates turning over to County at some point for maintenance. County has little resident expertise in this area.

Evaluator: Kara Blasco Comment: MultipleSystems Inc. and their partner companies are apparently very aware of the needs of this type of project. However, there is one area in which their approach differs from what we envision. Repeatedly, there is reference to a single or "the" technology vendor. We see the very strong likelihood that the best final product will be realized by acquiring technology from a variety of vendors, picking and choosing the items that are best for us and the vendor that seems best suited for our needs. I would see this as one of the most important functions and strangely out of place in an otherwise amazingly detailed and comprehensive proposal.

Criteria: The proposal showed knowledge of potential technical, regulatory, economic, environmental, or other issues that will be addressed by the technical approach

Evaluator: Kara Blasco Comment: Obviously the Multiplesystems Inc. team, especially the Principle in Charge John Poloski, is on the leading edge of the industry edge of the industry's effort to establish technical and regulatory guidelines that this rapidly developing application of ITS technology to transit requires. Their proposal provides a extensive reference to the process of regulatory development, the base from which it is developing and the progress to date. John Poloski has been and still is a prominent figure in this effort.

This report consists of 30 more pages. For the purpose of this brochure the report is truncated here.

3.30 2.96 3.80 0.00

Good Good Excellent N/A

County of USA Department of General Services - Purchasing Division


Oral and Risk Summary Report

Project Name: RFP-CR-003 Technical Assistance Project Project Number: 0000000144

Proposal: ATI Number of evaluators who completed Oral Evaluations: 4 out of 5 and number of evaluators who completed Risk Evaluations: 5 out of 5

Proposal: MultipleSystems Inc. Number of evaluators who completed Oral Evaluations: 4 out of 5 and number of evaluators who completed Risk Evaluations: 5 out of 5

Ranking Based on Oral Evaluation Score

Proposer's Name: ATIOral Evaluation Score: 12.50 Risk Evaluation Score: 8.25 Less Risk Score (MAX Risk - Risk EVAL Score): 27.75

ORAL Less RISK

Proposer's Name: MultipleSystems Inc.Oral Evaluation Score: 10.00 Risk Evaluation Score: 12.25 Less Risk Score (MAX Risk - Risk EVAL Score): 23.75

ORAL Less RISK

Overall Caucus Comments:

Proposer's Name: ATIOral Evaluation Score: 12.50 Risk Evaluation Score: 8.25

Personnel

Score: 6.50

Past Performance

Score: 1.00

Team Issues

Score: 2.00

Approach

Score: 2.00

Technical Response

Score: 1.00

Caucus Comments:

Evaluator Comments - ORAL

Personnel

Criteria: Has the key management been identified?

Evaluator: Samuel JohnsonComments: Appears to be very qualified and experienced. I believe they have the capabilities to make this project a success.

Evaluator: Kara BlascoComments: Assigned project manager is a recognized expert and leader in the integrated technology field.

Evaluator: John TishComments: Key management has been identified. Project manager is a leader in this field.

Criteria: Have the key consultants and other team members been identified?

Evaluator: Samuel JohnsonComments: I don't believe this question is applicable.

Evaluator: Kara BlascoComments: Good presentation on approach to the work, chain of command, use of staffing resources.

Evaluator: John TishComments: They did not guarantee that the three individuals in each of the specialty areas would be assigned for the entire project. Not all of the team members were present.

Criteria: Has the presentation been clear as to the role and responsibilities of key management and consultants and team members?

Evaluator: Samuel JohnsonComments: All of the members of the Avail team have specific responsibilities. all are centered around the project manager.

Criteria: Will the project manager and other presenter(s) be able to communicate with elected officials, staff, and the public

Evaluator: Samuel JohnsonComments: Pete Bennett speaks very well.

Evaluator: Kara BlascoComments: Project Manager is frequent presenter and recognized authority at industry conferences and is an excellent public speaker and one to one communicator.

This report continues for another fifteen pages. It includes evaluator comments for oral and risk evaluation. In order to keep this brochure short

and simple the report is truncated at this point.

County of USADepartment of General Services - Purchasing Division


Overall Combined Report Date: 1/8/2004 Time: 8:44:10 PM

Project Name: RFP-CR-003 Technical Assistance Project Project Number: 0000000144

Ranking based on Initial Written + Oral Score

Proposal: ATI

Overall Score=15.82 Risk Evaluation Score: 8.25


Overall Score=13.35 Risk Evaluation Score: 12.25

Ranking based on Initial Written + Oral + Risk Score (75% of Overall Initial Written & Oral + 25% of Less Risk)

Proposal: ATI

Overall Score with Risk =18.80 Overall Score without Risk =15.82 Risk Evaluation Score: 8.25


Overall Score with Risk =15.95 Overall Score without Risk =13.35 Risk Evaluation Score: 12.25

Proposal Awarded: ATI

Award Notes:

Proposer's Name: ATICombined Overall Score: 15.82 Risk Evaluation Score: 8.25Initial Written Evaluation Score: 3.32 Oral Evaluation Score: 12.50 Risk Evaluation Score: 8.25

Oral Score

Personnel

Score: 6.50

Past Performance

Score: 1.00

Team Issues

Score: 2.00

Approach

Score: 2.00

Technical Response

Score: 1.00

Initial Written Score

Initial Technical

Score: 3.71

Initial Personnel

Score: 3.25

Initial Past Performance

Score: 2.85

Proposer's Name: MultipleSystems Inc.Combined Overall Score: 13.35 Risk Evaluation Score: 12.25 Initial Written Evaluation Score: 3.35 Oral Evaluation Score: 10.00 Risk Evaluation Score: 12.25

Oral Score

Personnel

Score: 5.25

Past Performance

Score: 0.75

Team Issues

Score: 1.75

Approach

Score: 1.50

Technical Response

Score: 0.75

Initial Written Score

Initial Technical

Score: 3.30

Initial Personnel

Score: 2.96

Initial Past Performance

Score: 3.80

eRFP Configurator

You can customize eRFP evaluation process with the configurator.

You can register and select detailed criteria for your evaluation project template

You can specify the evaluation process in your project template

You can also customize the formula and rating system used in your project template

eRFP Builder

You can easily create the RFP through the template based RFP builder

A strategic solution for purchasing organizations

Technology

Transcript of A strategic solution for purchasing organizations