Mirones, Daniel Sent: Friday ...€¦ · 1301 International Parkway, Suite 300 | Fort Lauderdale,...

1

From: Mirones, Daniel <[email protected]>Sent: Friday, March 18, 2016 1:15 PMTo: Leznoff, JoAnne; [email protected]; Cc:; Potvin, Denise; Topp, Bruce; McCabe, Shelley; Potlock, Stuart;

[email protected]: Eppig, Brian; Barba, Mike; Hernandez, Anthony (Philadelphia Principal); Turner, MelissaSubject: Florida PALM IV&V - Reissue of February Monthly IV&V Assessment ReportAttachments: Florida PALM Monthly IVV Assessment - February 2016 v2 0.pdf

Grant Thornton is reissuing the February Monthly IV&V Assessment Report, revising our summary risk assessment for the project from “Red” to “Yellow.” The attached version 2.0 supersedes the version originally submitted on March 10.

Since the original release of our report on March 10 we have responded to requests for clarification regarding the artifacts used in assessing the project’s governance and the impact of the revised governance charter. After considering the substance of these requests for clarification we have determined that our original findings are not sufficiently clear with respect to the project documents submitted with the budget amendment in February. In particular:

1. The Florida PALM Project Team submitted and revised its PMP and governance charter on February 19.

2. Although the changes made to the project’s governance effectively resolved issues in how the project addressed Proviso requirements, the project steering

committee as defined in the revised governance charter has not convened.

3. Project changes requiring steering committee review were reviewed and approved in February consistent with the project’s previous governance, not the revised

governance.

Because of the discrepancy between the governance charter of February 19, which has been approved but not implemented, and the actual governance observed by the IV&V team in February, certain language in the earlier Monthly IV&V Assessment Report executive summary may have led the reader to believe that the February 19 version of the governance charter was at odds with proviso. This is inaccurate.

To clarify the impact of the new governance charter on the overall risk to the project, we have therefore reissued our report, affirming the project team’s February 19 revisions to the PMP and governance charter while noting that the project governance charter is accepted but not fully implemented.

Daniel Mirones Grant Thornton LLP 1301 International Parkway, Suite 300 | Fort Lauderdale, FL | 33323 | United States T (direct) +1 305 794 1122 E [email protected] | W www.GrantThornton.com

Right-click here to download pictures. To help protect you r priv acy, Outlo ok prevented auto matic downlo ad o f this pictu re from the In ternet.GT Image

Grant Thornton LLP is the U.S. member firm of Grant Thornton International Ltd. Grant Thornton International Ltd and its member firms are not a worldwide partnership, as each member firm is a separate and distinct legal entity. In the U.S., visit Grant Thornton LLP at www.GrantThornton.com.

Please consider the environment before printing this email.

Please understand that, unless expressly stated otherwise, any written advice given by Grant Thornton LLP that is contained in, forwarded with, or attached to this e-mail is: (1) limited to the matters and potential tax consequences specifically addressed herein, and; (2) not intended or written by Grant Thornton LLP as advice on the application or potential application of any penalties that may be imposed under any federal, state, or foreign statute or regulation in any manner.

This e-mail is intended solely for the person or entity to which it is addressed and may contain confidential and/or privileged information. Any review, dissemination, copying, printing or other use of this e-mail by persons or entities other than the addressee is prohibited. If you have received this e-mail in error, please contact the sender immediately and delete the material from any computer.

Date: 3/18/2016

Version 2.0

Monthly Independent Verification and Validation Assessment Report

Florida Planning, Accounting and Ledger Management (PALM) Project

February 2016

Florida PALM Project IV&V Monthly Assessment Report 3/18/2016

Table of Contents

Executive Summary ............................................................................................................... 4

Key Project Indicators ........................................................................................................ 5

Assessment of Project Risk ............................................................................................... 5

Findings Closed This Assessment Period ......................................................................... 7

Summary Findings and Recommendations ....................................................................... 7

Detailed Assessment ........................................................................................................... 10

1. Project Execution - Mandated Criteria ......................................................................... 10

1.1 Scope Management and Change Control ............................................................. 10

1.2 Schedule Management .......................................................................................... 11

1.3 Cost Management .................................................................................................. 13

1.4 Risk Management .................................................................................................. 15

1.5 Issue Management ................................................................................................. 17

2. Project Governance ..................................................................................................... 18

2.1 Governance ............................................................................................................ 18

2.2 Decision Management ........................................................................................... 20

3 Project Execution – Additional Criteria ......................................................................... 21

3.1 Performance Management .................................................................................... 21

3.2 Staffing Management ............................................................................................. 23

3.3 Action Item Management ....................................................................................... 25

4 Procurement and Contracts .......................................................................................... 27

4.1 Procurement and Vendor Management................................................................. 27

5. Quality Management .................................................................................................... 30

5.1 Quality Management .............................................................................................. 30

5.2 Deliverable Management ....................................................................................... 32

5.3 Lessons Learned Management ............................................................................. 33

6. Communications and Outreach ................................................................................... 35

3


6.1 Communications Management .............................................................................. 35

6.2 Stakeholder Management ...................................................................................... 36

6.3 Content Management ............................................................................................ 38

6.4 Collaboration Management .................................................................................... 39

Appendix 1. Project Element Value Assessment and Mapping .......................................... 41

Value Map Approach ....................................................................................................... 41

Process Values ................................................................................................................ 42

Appendix 2. Data Management Plan Assessment Summary ............................................. 43

Summary of Deliverable Review Report .......................................................................... 43

Updates to Deliverable Review Report ............................................................................ 44

Appendix 3. Interviews and Meetings Summary ................................................................. 45

Appendix 4. Project Documentation Reviewed ................................................................... 46


Executive Summary

Grant Thornton is independently assessing the Florida PALM Project, providing observations and recommendations on the program management processes in place to manage, monitor and control the Florida PALM Project through its current phase, known as Pre-Design, Development and Implementation (Pre-DDI). This report presents the technical, schedule and cost risks identified by the Grant Thornton Independent Verification and Validation (IV&V) team for the Pre-DDI phase of the Florida PALM project during the month ending February 29, 2016.

The IV&V team as an unbiased reviewer has independently reviewed key project performance indicators and assessed project risk utilizing both mandated criteria defined in the IV&V scope of services as well as Grant Thornton’s project assessment framework. Based on this review, the IV&V team assesses the risk status of the Florida PALM Project as “Yellow,” indicating that the project currently faces a set of challenges that could, if left unmanaged, negatively impact the project’s outcome in terms of schedule, budget or quality.

Overall risk status of Florida PALM project:

During this assessment period the project team updated it’s project management plan (PMP) and Governance Charter, This update serves as the basis for closeout of our previous highest-priority findings.

To align with draft FY16/17 Proviso, the Florida PALM project team received approval to delay the anticipated release of the Software and System Integrator (SSI) Invitation to Negotiate (ITN) for a minimum of 27 weeks and a maximum of 40 weeks. The project team received approval to delay the acceptance date of the Florida PALM project’s Data Management Plan (DMP) by almost 14 weeks. These changes to the project timeline – particularly the delay of the ITN release – represent significant modifications to the project’s master schedule.

Y

5


Key Project Indicators

Status Image

Is the project on schedule? YES

Is the project expected to remain on schedule? YES

Is the project currently within budget as defined in the spend plan? YES

Is the project expected to remain within budget as defined in the spend plan? YES

Were any project changes proposed or approved during this reporting period? YES

Assessment of Project Risk

Throughout the assessment, Grant Thornton uses a red-yellow-green indicator to evaluate the current status of each area and metric. The table below contains an explanation of each status indicator:

Image Description

Red. The assessment category or area faces a challenge or set of challenges that

threatens the project’s outcome in terms of schedule, cost or quality. The project team should take corrective action immediately.

Yellow. The assessment category or area faces a challenge or set of challenges that

could, if left unmanaged, negatively impact the project’s outcome in terms of schedule, cost or quality in the future. The project team should prioritize corrective action.

Green. The assessment category or area is on track without material issues. The

project team should consider any recommendations offered by the IV&V team as process improvement opportunities only.

The IV&V team utilizes the lowest indicator in each sub-category to determine the summary risk rating. A more detailed view of each of the elements that comprise each sub-category can be found in the Detailed Assessment section beginning on page 10 of this assessment report.

Project Area Current

indicator January

2016 December

2015

1. Project Execution - Mandated Criteria 1.1. Scope management and change control 1.2. Schedule management 1.3. Cost management 1.4. Risk management 1.5. Issue management

2. Project Governance 2.1. Governance 2.2. Decision management

R

Y

G

Y

Y

Y

Y

R Y

6


Project Area Current

indicator January

2016 December

2015

3. Project Execution - Additional Criteria 3.1. Performance management 3.2. Staffing management 3.3. Action item management

4. Procurement and Contracts 4.1. Procurement and vendor management

5. Quality Management 5.1. Quality management 5.2. Deliverable management

6. Communications and Outreach 6.1. Communications management 6.2. Stakeholder management 6.3. Content management 6.4. Collaboration management

Project Execution – Mandated Criteria. This is the broadest category of project elements and contains distinct, mandated criteria from the RFQ for IV&V services. There are 21 elements assessed in this project area, ranging from scope management to schedule, cost, risk and issue management. No significant issues were noted in 20 out of the 21 areas. No changes were noted this assessment period. Scope management remains in “Yellow” status driving the summary indicator for this project area to a “Yellow” status.

Project Governance. Of the seven project elements examined within the area of project governance, two previous “Red” indicators have been changed to green to reflect legislative endorsement of the revised governance charter submitted with the budget amendment on February 19.

Project Execution – Additional Criteria. Risk indicators for the sixteen elements examined within this project area have not changed since the last assessment period. Three findings in this area are driving a risk indicator of “Yellow.” There are no “Red” ratings for this area.

Procurement and Contracts. All ten elements related to the procurement and contracts project area indicate a “Green” status.

Quality Management. Of the twelve project elements related to quality management, all but one indicates a “Green” status. The remaining element indicates a “Yellow” status and is related to the project’s Quality Management System (QMS), which as of the end of this reporting period remains incomplete. There are no “Red” ratings for this area.

Communications and Outreach. The IV&V team has noted improvement in one of the twelve project elements related to project communications and outreach, notably due to the successful adoption of the project’s communications plan and quarterly reporting of their collaboration activities. Of the remaining elements indicating a “Yellow” status, we have made specific recommendations to address these issues in due course. There are no “Red” ratings for this area.

Y

Y

G

G

G

G

Y

Y

Y

Y

Y

Y

7


As part of the IV&V process for assessing and reporting on identified risks, we have applied a value assessment to each area which guides the determination of the priority, or relative impact an individual observation can have on the entire project. A description and explanation of this value assessment approach and the most recent results can be found in Appendix 1 beginning on page 41.

Findings Addressed This Assessment Period

The following table summarizes all findings that have been addressed this assessment period.

Summary Finding Current

indicator Area(s)

impacted New Observation

F01. Although a governance

structure is in place, the project is vulnerable to reversals of previous decisions such as the exclusion of payroll from the ITN (Decision 4, reversed in Decision 87). Until an appropriate governance structure is in place, the project team cannot obtain appropriate approvals for project changes.

)

1.1 – Scope management and change control

2.2 – Decision management

During this assessment period the project team obtained legislative endorsement for a governance structure that satisfies Proviso requirements as well as the needs of the project. Finding 12 has been added related to the implementation of the updated governance structure.

F02. The project's governance

structure is at odds with the Legislative interpretation of Proviso.

2.1 – Governance

During this assessment period the project team obtained legislative endorsement for a governance structure that satisfies Proviso requirements as well as the needs of the project.

F11. The collaboration strategy

for FY15-16 adopted by the project team does not include specific metrics.

6.4 – Collaboration management

As part of its collaboration strategy rollout, the project team established partner collaboration metrics such as participation and level of engagement and is evaluating them on a quarterly basis.

Summary Findings and Recommendations

The following table summarizes any “Red” or “Yellow” findings from the detailed assessment section of the report and maps our associated observations/recommendations to the project health indicator presented in our overall risk reporting:

G

G

G

8



indicator Area(s)

impacted Recommendation

F03. The process for tracking,

measuring, and reporting several key project performance metrics such as SPI and CPI has not been formalized.

3.1 – Performance management

Formalize the process for tracking, measuring and reporting key project performance metrics by implementing the QMS. The project team may also consider mapping key project metrics, their location, and the process by which they are being calculated as an interim reference.

F04. The lack of a full time OCM

manager places increased demands on the project director who is temporarily filling that role.

3.2 – Staffing management

The project team should fill the OCM manager position and continue to follow its staffing plan, or consider alternatives that will relieve the burden on the project director.

F05. The project team tracks

open action items and reviews them on a regular basis in the RAIDL meetings and the weekly RADAR report. There are no separate metrics tracked.

3.3 – Action item management

Identify action item health metrics such as percent closed by due date and average aging, and add these to regular (monthly or quarterly) internal project review processes.

F06.The project team currently

relies on high-level process definitions and standards outlined within the PMP to manage project quality instead of following a formalized deliverable lifecycle development and review process.

5.1 – Quality management

5.2 – Deliverable management

Implement a full deliverable lifecycle development and review process. The project team should consider including this process in the QMS.

F07. The updated

Communications Plan adopted in January contains a full stakeholder assessment, but does not include feedback or monitoring processes on the effectiveness of the communications.

6.1 – Communications management

The project should consider including discrete processes for monitoring the effectiveness of communications.

F08. The project team currently

tracks participation in events but does not have a formal strategy or metrics for measuring the other key elements of change such as awareness and buy-in.

6.2 – Stakeholder management

The project should consider creating and implementing additional tools for collecting, measuring, and reporting stakeholder awareness and buy-in.

F09. The OCM team has created

and published document management standards and naming conventions. There is ambiguity around how to track versions of deliverables and other maintained project artifacts.

6.3 – Content management

The project should complete the implementation of the QMS, which has a segment identified to address this topic.

Y

Y

Y

Y

Y

Y

Y

9



indicator Area(s)

impacted Recommendation

F10. Each team has their own

SharePoint file structure, leading to confusion and difficulty in finding documents across tracks.

6.3 – Content management

The project should consider creating a standardized SharePoint file structure and taxonomy for common elements within each track (e.g. status, work plans, deliverables), while enabling flexibility for track-specific content.

F12. Until the revised

governance structure is implemented, the project is vulnerable to reversals of previous decisions such as the exclusion of payroll from the ITN (Decision 4, reversed in Decision 87).

1.1 – Scope management and change control

2.2 – Decision management

To avoid future reversals of key decisions made by the project, implement the February 19 updated governance structure.

Y

Y

10


Detailed Assessment

1. Project Execution - Mandated Criteria

1.1 Scope Management and Change Control

Key Metrics

Item Observed Current Status

Previous Status Notes

Number of open change requests

0

Change requests continue to follow the documented change request process as outlined per the project management plan (PMP). Number of change

requests processed this period with final determination

4

Number of open change requests in overdue status

0

Process Observation Summary

Change requests continue to follow the documented change request process as outlined within the PMP. During the current assessment period, final dispositions were determined for four Project Change Requests (PCR), PCR 15, PCR 16, PCR 17 and PCR 18. The most significant project changes involve moving the dates for key project deliverables leading up to the ITN as well as aligning the earliest date for the release of the ITN with applicable proviso language.

G

G

G

N/A

G

N/A

11


Assessment Category Findings and Observations Recommendations Status

Determine that a structured process is in place for identifying, documenting, reviewing, and obtaining appropriate approval for any change to project scope.

Until the revised governance structure is implemented, the project is vulnerable to reversals of previous decisions such as the exclusion of payroll from the ITN (Decision 4, reversed in Decision 87).


Current

Previous

Trend

Verify that pending project changes and decisions are clearly communicated to impacted Stakeholders in the appropriate level of detail.

The project team generates periodic status reports, which include details regarding pending changes and decisions. These status reports are distributed at project meetings involving stakeholders such as the project steering committee (PSC) and periodic meetings with AST and the legislature.

No new recommendations. Current

Previous

Trend

Verify that any changes to the project scope adhere to the defined change control processes and the appropriate governance levels for change process.

The project continues to follow the change control processes outlined within the PMP.


Previous

Trend

1.2 Schedule Management

Key Metrics



% Complete 82%

Revision of schedule reviewed was version 90 dated 02/24/2016, which was the last version of the master schedule posted during the assessment period.

Number of Late Tasks

24

Y

Y

G

G

G

G

G

G

G

G

12



The Florida PALM Project Team continues to follow the steps outlined in the PMP and the Work Tracks are planned throughout the Pre-DDI phase. Regular updates to the master schedule were made during the assessment period. Changes to the schedule were documented, and were applicable, schedule changes cited the applicable PCR.

Over half of the late tasks reported in the master schedule are due directly to delays in the release of project funding. The remaining tasks are related to future events or milestones that are addressed by release of project funding as anticipated or a re-baseline of the schedule. The IV&V team would expect that these late tasks would resolve with the next iteration of the schedule baseline.


Identify whether the project is on schedule. If any schedule slippage exists, identify whether project leadership has assessed severity of the slippage, and identified mitigating actions to address.

The master project schedule currently indicates an on-track status. Schedule performance lags slightly but is within an acceptable tolerance.


Previous

Trend

Determine if project milestones have been identified and documented.

Project milestones continue to be called out in the master project schedule.


Previous

Trend

Determine if a process exists for comparing project progress against planned (baselined) measures.

The project continues to review the Project Schedule against baseline on a regular basis as evidenced by the weekly status reports.


Previous

Trend

G

G

G

G

G

G

13


1.3 Cost Management

Key Metrics



Cost Performance Index (CPI)

1.02

The project is currently tracking CPI in the master project schedule.


The project continues to follow their cost management procedures as outlined in the PMP and is continuing to refine the level of detail needed. During this assessment period the project team modified the definition of actual spend to be costs incurred instead of invoices paid. The project team continues to track schedule performance (SPI) and cost performance (CPI) using earned value management (EVM) techniques against the master schedule. Proxy values for project cost are used.


Determine whether the project budget appears sufficient to support the activities associated with the scope of the project.

The project budget for the year appears sufficient based on the Spend Plan and the list of two-party agreements.


Previous

Trend

Determine the project's compliance with the defined cost management techniques in the PMP (cost management plan component).

The project team follows the cost management process outlined in Appendix 2 of the PMP.

The project team may consider elaborating on the spend baseline and spend control descriptions in the Cost Management section of the PMP.

Current

Previous

Trend

G

G

G

G

G

G

14



Verify that a project baseline has been established. Assess the projected costs approved and the actual costs of implementing and maintaining the completed project against the baseline.

The project spend plan currently identifies planned expenditures fiscal year to date. Beginning in this assessment period the spend plan was changed to report and compare incurred liabilities each month instead of cash payments.

The project team should update Appendix 2 of the PMP with the new process being followed as a result of reporting on incurred liabilities instead of cash payments.

Current

Previous

Trend

Verify that monitoring is being done on progress against project costs and expended effort.

Cost monitoring is being performed in accordance with the description of spend control in Appendix 2 of the PMP.

The project team should consider elaborating on the spend baseline and spend control descriptions in Appendix 2 of the PMP.

Current

Previous

Trend

Verify that the right performance measures are being collected and managed for cost management.

Beginning in this assessment period, the spend plan was changed to report and compare incurred liabilities each month instead of cash payments.

The project team should update Appendix 2 of the PMP with the new process being followed as a result of reporting on incurred liabilities instead of cash payments.

Current

Previous

Trend

G

G

G

G

G

G

15


1.4 Risk Management

Key Metrics



# of risks opened during this period

5 Volume of new risks added to the risk log

and risks closed are appropriate for the size and complexity of this project. # of risks closed

during this period 3

# of overdue risks 0

There were no risks observed with a due date that occurs in the past.

# of risks transitioned into issues

0

There were no risks transitioned into issues.


Risks on the Florida PALM Project continue to be tracked and managed using the MS SharePoint portal following the risk management processes as outlined per the PMP. These risks are reviewed at regular RAIDL meetings. AST continues to attend the project team’s bi-weekly RAIDL meeting for observation and to provide feedback when solicited by the project team.


Verify that a project Risk Management Plan and supporting procedures are in place and complete.

The Risk Management process is defined in Appendix 9 of the PMP.


Previous

Trend

Verify that the Risk Management plan is being followed and is effectively identifying, tracking, and managing project risk in a standardized fashion.

Risk management meetings are held at least weekly, and evidence of the identification, tracking and managing of project risk occurs in the project team’s risk log.


Previous

Trend

G

N/A

G

N/A

G

N/A

G

G

G

G

G

G

16



Determine whether the Executive Sponsorship understands effective risk management planning and execution and are reinforcing the importance of sound risk management in their communications with project personnel.

Results of the risk reporting function of the project team are brought to the attention of the Project Steering Committee at least monthly and discussed as evidenced by meeting agendas and by direct observation.


Previous

Trend

Determine whether project team members possess a command of the defined risk management plan and processes.

Risk management processes were followed by each of the Work Tracks as well as by the PMO during the assessment period.


Previous

Trend

Verify that consistent criteria for evaluating and quantifying risk likelihood and severity levels have been defined.

A criteria for evaluating and quantifying likelihood and severity is performed by the risk management team in accordance with the risk analysis process defined in Appendix 8 of the PMP.


Previous

Trend

Verify that risks have been prioritized for mitigation.

It was observed that the project team holds regularly scheduled RAIDL meetings to identify and address risks in accordance with Appendix 8 of the PMP.

Continue to adhere to the regular RAIDL review process and monitor and address identified risks in a timely manner and continually improve the process.

Current

Previous

Trend

G

G

G

G

G

G

G

G

17


1.5 Issue Management

Key Metrics



New issues opened this period

0

There were no new issues opened this period.

Issues closed this period

0

There were no issues closed during this period.

Open Issues past target completion

4

All open issues currently past the target completion date are related to project funding, up from 3 issues for the last reporting period.


Issues on the Florida PALM Project continue to be tracked and managed using a MS SharePoint portal. The project actively manages project risks and has had very few of these risks become issues. The project issue database is used for issues that have an active impact on the project scope, schedule or execution. AST continues to attend the project’s bi-weekly RAIDL meeting for observation and to provide feedback when solicited by the project team. Currently overall project status and budget trends are being reported as “Red” due to Issues 7, 8, 9 and 11, which all stem from the lack of released funding.


Verify that guidelines to determine which issues are subject to a formal evaluation process have been established and maintained.

Issue management is formally defined in Appendix 11 of the PMP. Exhibit 11-12 defines the issue escalation levels.


Previous

Trend

Verify issues are collected and analyzed, and corrective actions have been created to address them.

Corrective actions included for all issues captured in the issue log (RAIDL). Issue review occurs on a regularly scheduled basis per the outlined process.


Previous

Trend

G

N/A

G

N/A

G

G

G

G

G

G

18



Verify the existence and institutionalization of an appropriate project issue tracking mechanism that documents issues as they arise, enables communication of issues to proper Stakeholders, documents a mitigation strategy as appropriate, and tracks the issue to closure.

The project utilizes the Microsoft SharePoint issue management log in accordance with the issue management process defined in Appendix 11 of the PMP.


Previous

Trend

Verify that corrective action has been taken on identified issues.

Corrective actions are taken on identified issues through the weekly and bi-weekly RAIDL process.

The project team may consider moving issue review to a weekly activity as the volume of open issues increases.

Current

Previous

Trend

2. Project Governance

2.1 Governance

Key Metrics



N/A N/A

The individual metrics associated with project governance processes will be tracked within those specific sections of the document.


As part of the current legislative session, additional guidance regarding the governance structure is anticipated. The project team is working to proactively incorporate this feedback into an updated version.

G

G

G

G

N/A N/A

19



Evaluate effectiveness of project governance for a project of this size and complexity.



Previous

Trend

Assess executive Stakeholder buy-in, participation, support and commitment, and that open pathways of communication exist among all Stakeholders.


As the new project steering committee is convened, the project team should consider actively engaging the new members.

Current

Previous

Trend

Verify that lines of reporting and responsibility provide adequate technical and managerial oversight of the project.

The current governance charter establishes clear lines of reporting for the project team. Under the current organization structure, oversight defaults to the Executive and Project Sponsors.

The project team should consider including oversight roles in the replacement project charter document, as well as a description of the oversight roles performed by independent bodies such as the AST and IV&V teams.

Current

Previous

Trend

R

G

R

G

G

G

20


2.2 Decision Management

Key Metrics



Number of new decisions added this period

0

There are no new decisions added to the decision log this period.

Number of decisions closed this period

0

There were no decisions closed this period.

Number of overdue open decisions

0

There are no overdue open decisions.


The processes associated with decision management are clearly defined in terms of required information, approvals, and escalations. The decision log is reviewed regularly and contains a sufficient level of detail appropriate for a project of this size and complexity. An accounting is included for each closed decision in accordance with the PMP.


Verify that Stakeholders have bought in to all changes which impact project objectives, scope, cost, or schedule.

Until the revised governance structure is implemented, the project is vulnerable to reversals of previous decisions such as the exclusion of payroll from the ITN (Decision 4, reversed in Decision 87).


Current

Previous

Trend

Verify that the project team and project sponsors have reviewed and approved any project changes that may impact general scope or direction to help ensure alignment with the project objectives.

As defined in the current project charter, the project steering committee and project sponsors review project decisions identified for steering committee and CFO review as they are made and presented as evidenced by documented decisions and changes to the project made during the assessment period.


Previous

Trend

G

N/A

G

N/A

G

N/A

G

Y

G

G

21



Determine whether the project and project sponsors have reviewed and approved any deviations from the project baseline budget prior to further action (e.g., contract modification).

As defined in the current project charter, the project steering committee and project sponsors review project decisions identified for steering committee and CFO review as they are made and presented as evidenced by documented decisions and changes to the project made during the assessment period.


Previous

Trend

Determine whether the baseline (and any re-baselines) to the master project schedule was vetted through and received approval from project sponsors.

The current baseline for the master schedule was approved by the project steering committee. During this assessment period, other schedule changes were authorized for which a new baseline is expected to be vetted.


Previous

Trend

3 Project Execution – Additional Criteria

3.1 Performance Management

Key Metrics



N/A N/A

The individual performance metrics for the project are identified and captured in each specific section (e.g. cost, schedule, etc.).


The project continues to formalize how it reports on the performance metrics defined within the PMP. The project is working with the IV&V Vendor to develop an implementation plan for the Quality Management System (QMS), which will include refinements to the performance metrics and implementation of project support tools to enable easier collection and reporting of project performance. The implementation of additional metrics is expected to last into May 2016.

G

G

G

G

N/A N/A

22



Evaluate implementation of the performance management activities and metrics to determine if processes are effective and are being followed.

The process for tracking, measuring, and reporting several key project performance metrics such as SPI and CPI has not been formalized.

Implement the project performance metrics section of the QMS.

The project team may consider mapping key project metrics, their location, and the process by which they are being calculated as an interim reference.

Current

Previous

Trend

Evaluate if appropriate mechanisms are in place for project self-evaluation and process improvement.

Lessons learned are captured after each task and tracked in SharePoint.


Previous

Trend

Verify that selected measurements are being obtained and analyzed to manage the project and support organization needs.

Key project metrics are being measured and reported individually.


Previous

Trend

Verify that the project’s performance is periodically reviewed and aligned with current and anticipated needs, objectives, and requirements of the organization, customers, and end users as appropriate.

Project performance is reviewed during the regular internal status meetings. Additionally, project performance is reported externally on a weekly and monthly basis.


Previous

Trend

Y

Y

G

G

G

G

G

G

23


3.2 Staffing Management

Key Metrics



Percent of scheduled positions filled.

95% (18 of 19)

The Project has not filled the OCM lead due reportedly to the uncertainty raised by delayed funding release.

Turnover 5.3% (1 of 19)

One project Team member has left the project in the past six months. The individual who left went to a position in DFS where they will continue to support the project. The position has been filled with a resource from DFS who was previously supporting the project.


The PMO manages a formal staffing and onboarding process for state staff and the Contract Managers manage the process for their assigned Contractors. Onboarding procedures and support materials have been developed and are shared with new members of the team to provide a clear and consistent background on the project and operating guidelines moving forward.

The project team looks for the best possible candidate for each position, from both a skillset and personality fit. This has led to longer lead times to fill some positions, but has resulted in a stronger, more capable Project Team. The most recent attempts to hire an OCM Manager have been hindered by uncertainty caused within the project by the delay in the budget funding release. Other project positions are forecast to be filled later in the year with dependencies on the ITN release and evaluation dates.


Verify that project roles and associated responsibilities are clearly defined, documented, and communicated and verify that each project team member is mapped to a distinct project role.

The current project charter establishes clear lines of reporting for the project team. Under the current organization structure, oversight defaults to the executive sponsor and project sponsors.


Previous

Trend

Y

Y

G

G

G

G

24



Determine whether the design and structure of the project management team is sufficient to support the project given the project's size, scope and complexity.

The structure and staffing of the PMO team continues to be appropriate for the Pre-DDI effort.


Previous

Trend

Determine whether the design of the organizational structure reflects clear lines of reporting, responsibility and accountability.

The project’s organization structure is clear and the team members understand their reporting relationships and how they support each other.


Previous

Trend

Determine whether the project operates in accordance with the organizational structure, wherein the defined/documented lines of reporting, responsibility and accountability are practiced/adhered to by project team members.

The project team continues to follow the project structure with each track lead managing their team.


Previous

Trend

Determine whether project resources appear appropriately qualified to perform their assigned duties (based on the defined project roles/responsibilities).

The current project resources appear to have the background experience and knowledge required for their positions.


Previous

Trend

G

G

G

G

G

G

G

G

25



Determine the project's compliance with the resource management techniques defined in the PMP.

The project team follows the staffing planning and onboarding processes outlined within the PMP.


Previous

Trend

Evaluate the stability of the project team and cross sharing of knowledge to buffer for vacations and possible turnover.

The project team actively manages staffing and cross-sharing of information within each work track where possible.


Previous

Trend

Determine if adequate time and resources are assigned to each project phase to gauge whether the project is sufficiently "resourced" (time, money and staffing) for future work.

The lack of a full time OCM manager places increased demands on the Project Director who is temporarily filling that role.

The project should fill the OCM manager position and continue to follow its staffing plan based on the reality of the delayed budget amendment and the withholding of funds.

Current

Previous

Trend

3.3 Action Item Management

Key Metrics



Action items past due

0

There are no past due action Items.

Average age of past due items

0 days

G

G

G

G

Y

Y

G

G

G

G

26



The project has a strong discipline around identifying, assigning, tracking, and following up on action items. Each open Action Item is reviewed at each track’s regular (weekly or bi-weekly) RAIDL (Risk, Action Item, Decision, and Lessons Learned) meeting. At the bi-weekly PMO RAIDL meeting, the team reviews all of the open Action Items to validate their status.


Verify that Action Items are clearly defined and differentiated from project tasks, issues, and risks.

Action items are defined in the PMP and are distinct from project tasks, issues and risks.


Previous

Trend

Verify that Action Items are managed in a consistent manner across the project and are at an appropriate level of detail.

Action items recorded during the assessment period are consistent in level of detail.


Previous

Trend

Verify that open Action Items are regularly reviewed to ensure that they are being managed, and that if there is a change or larger impact, they are transferred to the master project schedule where appropriate.

The project team reviews action items at least weekly at regularly scheduled RAIDL meetings.


Previous

Trend

G

G

G

G

G

G

27



Verify that metrics are in place and reviewed on a regular basis regarding the status and closure of Action Items.

The project team tracks open action items and reviews them on a regular basis in the RAIDL meetings and the weekly RADAR report. There are no separate metrics tracked.

Identify action item health metrics such as percent closed by due date and average aging, and add these to regular (monthly or quarterly) internal project review processes.

Current

Previous

Trend

4 Procurement and Contracts

4.1 Procurement and Vendor Management

Key Metrics



% Contracts on Schedule.

100%

Current contracts are on schedule for deliverable or hourly service delivery.

% Contracts on Budget.

100%

Current contracts are within planned budget.

Contract Execution on Schedule.

Yes

Current planned procurements are on schedule.


Procurements and Vendor contracts are managed through the PMO. The project has two designated procurement professionals who manage the procurements from the initial strategy, to contract execution, and through contract closeout. There have been no significant changes to the procurement processes during the reporting period.

Y

Y

G

G

G

G

G

G

28



Determine if procurement has explicitly mapped the procurement objectives and strategy to the program objectives, and if such mapping accurately reflects the procurement strategy being followed.

The PMO manages procurements following the process outlined in the PMP and executes each work track’s Procurement Strategy as outlined.


Previous

Trend

Determine if solicitation documentation clearly articulates measurable project objectives, and clearly identifies what (if any), project objectives are the responsibility of the Contractor.

No new procurements were executed during the reporting period.


Previous

Trend

Determine if the solicitation documentation provides adequate detail on the project functional requirements, and if the solicitation process offers bidders, an adequate opportunity for dialogue on functional requirements so that questions, ambiguities and misunderstandings can be rectified prior to contract award.

The plans for the ITN include time for questions and dialog to enable bidders to gain an understanding of the project functional requirements. No other procurements were worked during the assessment period.


Previous

Trend

Determine whether key project risks have been shared with potential Contractors. Determine whether significant drivers of risk (e.g., organization and business process change, limitations in number and skills of agency resources, number and complexity of legacy systems to be replaced, etc.) to the potential Contractors have been explained in such a way that bidders can address and mitigate them in their bid.

The plans for the ITN include time for questions and dialog to enable bidders to gain an understanding of the project functional requirements. No other procurements were worked during the assessment period.


Previous

Trend

G

G

G

G

G

G

G

G

29



Determine whether documented evaluation processes exist and are being consistently followed.

The BPS and Procurement Teams are currently planning the evaluation process for the ITN. No other procurements were worked during the reporting period.


Previous

Trend

Determine whether the Evaluation Team has received training and ongoing guidance necessary for them to evaluate bids in an effective and consistent manner.

No procurements were worked during the reporting period.


Previous

Trend

Determine if matters that pose a risk to project scope are promptly escalated to the appropriate parties, including Contracts.

As critical decisions and scope items for the ITN are raised, they have been documented and managed either as project changes or as project decisions. No other procurements were worked during the reporting period.


Previous

Trend

Determine if invoice review processes are established.

The project is following the DFS Contract Lifecycle Guide and the guidelines published by the DFS Division of Accounting & Auditing for invoice review


Previous

Trend

G

G

G

G

G

G

G

G

30



Determine if the defined quality assurance and deliverable approval processes are being followed in the review and acceptance of all contract deliverables.

Deliverable review and approval processes are being followed for all contract deliverables.


Previous

Trend

Evaluate Contractor change request and defect tracking processes.

There were no change requests during this assessment period from contractors or contractor work which resulted in a project change request.


Previous

Trend

5. Quality Management

5.1 Quality Management

Key Metrics



Vendor Service Quality Scores.

N/A

Not yet tracked but will be rolled out with QMS.

Deliverable Quality Scores.

N/A


Project Quality Improvement Index.

N/A



A high-level approach to quality management is outlined within the PMP. The project team has finalized the QMS implementation strategy and has begun work on the QMS implementation plan. During this assessment period the project team updated the QMS implementation plan and released draft definitions documents for each section of the QMS.

G

G

G

G

N/A N/A

N/A N/A

N/A N/A

31



Verify that project self-evaluations are performed and that measures are continually taken to improve the process.

Project self-evaluation was not scheduled to be performed during this assessment period.

The project team should consider conducting a self-evaluation post release of the ITN.

Current

Previous

Trend

Verify that QA has an appropriate level of independence.

QA is monitored by the QA/Risk Manager.


Previous

Trend

Verify that all process definitions and standards are complete, clear, up-to date, consistent in format, and easily available to project personnel.

The QMS Implementation Strategy has been finalized. Until the QMS is released, the project team follows high-level process definitions and standards outlined within the PMP.


Previous

Trend

Assess whether the roles and responsibilities are clearly defined and responsible personnel are identified related to quality assurance and deliverable approval processes.

Deliverable review responsibilities are clearly defined and communicated to appropriate project team members and reflected in the appropriate DED documents.


Previous

Trend

G

G

G

G

G

G

G

G

32



Verify that the QA organization monitors the fidelity of defined processes in all phases of the project.

Until the QMS is released, the project team follows high-level process definitions and standards outlined within the PMP.

The project team should implement the QMS.

Current

Previous

Trend

Verify that the quality of products produced by the project is monitored by formal reviews and sign-offs where appropriate.

Project deliverables are reviewed and formally signed off.


Previous

Trend

5.2 Deliverable Management

Key Metrics



Milestone slippage and/or schedule changes due to deliverable development or review.

4

Four changes in February (PCR 14, 15, 16 and 17) required several changes to schedule and extended development/review time for project deliverables.


The project follows a structured deliverable review process. Key elements of that process include that the development and review steps for each deliverable are included in the master project schedule, and that a deliverable expectations document (DED) is created for each, defining the scope, content, and review process for the deliverable. A review committee of project team staff and project sponsors is identified and assigned to each deliverable. Upon completion of the review and edit process the committee makes a recommendation for approval to the project director, who approves each deliverable.

Y

Y

G

G

G

G

33



Ensure there is a clear definition of project deliverables and that deliverables are managed in a consistent manner.

The deliverable process outlined in the PMP is now being followed for internal deliverables. One recent example of the process was the updated of the project’s communications plan.


Previous

Trend

Verify that the defined deliverable review process is consistently administered across the project.

The deliverable process outlined in the PMP is now being followed for internal deliverables in addition to external (contractor) deliverables. One recent example of the process was the updated of the project’s communications plan.


Previous

Trend

5.3 Lessons Learned Management

Key Metrics



Lessons Learned Application.

Process in place

The current lessons learned process outlined in the PMP and in use by the project collects lessons learned, but does not index them to readily allow a proactive search of items relevant to a specific project activity or work track.


‘Lessons Learned’ are defined in the PMP as any useful information or experience gained through the course of the project that can be applied to a later phase or project activity. Currently, only lessons learned, which have a profound impact on the track are being captured.

The project has a good discipline around identifying, and documenting findings. Significant Lessons are entered into the project SharePoint portal. New or active Lesson Learned are reviewed at each track’s regular RAIDL (Risk, Action Item, Decision, and Lessons Learned) meeting. At the bi-weekly PMO RAIDL meeting, any new lessons learned items are reviewed.

G

G

G

G

G

G

34



Verify that lessons learned, which could improve processes have been identified.

Lessons learned are documented in SharePoint by the tracks as they are identified and discussed at the regular RAIDL meetings. There is not a clear process for ensuring the lessons are shared between tracks. (No change from initial assessment)

The project team should consider updating the lessons learned portion of the RAIDL meeting and SharePoint record to include the track(s) and project activities where the lesson learned would apply. This will facilitate the ability for the tracks to review applicable lessons learned at the right time during the project lifecycle.

Current

Previous

Trend

Verify that lessons learned are managed in a consistent manner across the project and are at an appropriate level of detail.

A standard SharePoint tool is used to track the lessons learned. Because of the breadth of topics addressed, it can be difficult to know there is an applicable lesson to be applied to a new project activity.


Previous

Trend

Verify that project or phase closeout reports are created to provide a concise evaluation of the project. The closeout report should summarize at a high level, the project objectives, variances or deviations from the plan/baseline encountered, lessons learned and next steps.

No project phases were closed during the reporting period.


Previous

Trend

G

G

G

G

G

G

35


6. Communications and Outreach

6.1 Communications Management

Key Metrics



Awareness Rating N/A

The project is tracking participation in events by stakeholders, but there is no process to evaluate the effectiveness of project communications on the various stakeholder groups.

Awareness Rating Trend

N/A

The project is informally tracking agency awareness, but has not yet developed a specific measure for agency awareness that can be monitored over time.


The project has adopted an updated communications management plan, pulling together individual elements that had been separately developed by the OCM team into a single cohesive plan. The plan includes a stakeholder assessment as well as the planned communications metrics and channels. However, measures for the effectiveness of the communications are not included.


Determine whether the project communications plan identifies the correct audiences, contains approaches and strategies to maximize receipt of the targeted messages, and contains effective feedback mechanisms.

The updated communications plan contains a full stakeholder assessment and guidelines for reporting of communications activities, but does not include processes for monitoring or obtaining feedback on the effectiveness of the communications.

The project should consider including discrete processes for monitoring the effectiveness of communications.

Current

Previous

Trend

Y

Y

N/A N/A

Y

Y

36



Determine if the plan is being successfully executed, observing the plans 'in action'.

The current project communications plan outlines key communication and outreach activities. During the current assessment period, project artifacts were developed consistent with the communication plan and were available for review by the IV&V team. The OCM team also maintains a communication log that substantiates project communication consistent with the communication plan.

No new recommendations.

Current

Previous

Trend

6.2 Stakeholder Management

Key Metrics



Engagement Rating. N/A

The project is not currently tracking a metric for Stakeholder engagement.

Engagement Rating Trend.

N/A

The project is not currently tracking a metric for Stakeholder engagement.


The project proactively manages outreach to Agency Stakeholders through multiple media including website updates, directed communications about the project and planning and execution of a number of workshop and workgroup meetings. The project communications management plan includes a full stakeholder analysis. The OCM team tracks participation and monitors engagement in these sessions to identify and proactively engage agencies that may require additional support. Formal outreach to external parities is being managed as needed.

G

G

N/A N/A

N/A N/A

37



Determine if a stakeholder assessment has been performed for the project, and assess the completeness and appropriateness of the targeted audiences and messages.

The project includes a stakeholder assessment as part of the updated communications plan.


Previous

Trend

Determine whether the change management plan contains approaches and strategies to maximize adoption, assess and manage resistance, assess and increase awareness and effect a successful project implementation.

The project team has adopted an updated communications plan.


Previous

Trend

Verify that outreach efforts exist to ensure that all Stakeholders are aware of the impending change and know their role in the change process. Verify that communication provides appropriate channels for feedback.

The project team performs proactive communication to impacted stakeholders and uses feedback mechanisms such as surveys and email to gauge the effectiveness of project messages.


Previous

Trend

Review plan to determine how the team plans to measure the key components of change (e.g., awareness, buy-in, etc.).

The project team currently tracks participation in events but does not have a formal strategy or metrics for measuring the other key elements of change such as awareness and buy-in.

The project should consider creating and implementing additional tools and processes for collecting feedback and measuring the effectiveness of the full breadth of stakeholder engagement activities.

Current

Previous

Trend

G

G

G

G

G

G

Y

Y

38



Determine whether the project proactively identifies areas at risk for resistance and effects strategies to manage the potential resistance.

The project team continues to document risk and issue information on SharePoint and communicate with the impacted parties.


Previous

Trend

6.3 Content Management

Key Metrics



Compliance with Document Archive Policies by team.

N/A

Formal guidelines for document management and archive structure, which would guide this metric, are under development.

Compliance with controlled document processes.

N/A

Formal guidelines for document management and archive structure, which would guide this metric, are under development.


The project extensively uses a Microsoft SharePoint collaboration site for the tracking and managing of working documents and artifacts. Version control is enabled in SharePoint, providing the ability to track revisions to documents if necessary. The project team consistently uses SharePoint collaboration for the internal development of project deliverables and other project artifacts.


Verify that the project has a clear strategy for maintaining and managing project artifacts.

The project has a strategy which guides the management of project artifacts including work products, deliverables, and KPIs. (No change from initial assessment)


Previous

Trend

G

G

N/A N/A

N/A N/A

G

G

39



Assess the level of detail and adherence to document and deliverable management policies and procedures.

The OCM team has created and published document management standards and naming conventions. There is ambiguity around how to track versions of deliverables and other maintained project artifacts.

The project should complete the rollout and training on standard document version tracking and management. This may be accomplished by the planned QMS rollouts for Document Quality and Release Management.

Current

Previous

Trend

Assess the consistency and ease of finding and accessing project documentation.

Each team has their own SharePoint file structure, leading to confusion and difficulty in finding documents across tracks. The OCM team has begun to develop a consistent file structure and guideline for the project’s content.

The project should consider creating a standardized SharePoint file structure and taxonomy for common elements within each track (e.g. status, work plans, deliverables), while enabling flexibility for track-specific content.

Current

Previous

Trend

6.4 Collaboration Management

Key Metrics



Partner Engagement N/A

The project captures Partner participation in project events and meetings. Planned participation is evaluated on a quarterly basis as the planned activities and partner engagement for the following quarter are being planned.


The Florida PALM Project has identified external partners with whom they will require collaboration to be successful. Outreach to a number of these entities has already occurred either through individual project activities or through execution of the Communications Plan. A formal Collaboration Strategy for FY15/16 was adopted in November and is being followed. This strategy addresses planned engagement with key external Stakeholders including State Agencies, the FFMIS entities, other Public Sector ERP Implementations, and external interest groups.

Y

Y

Y

Y

N/A N/A

40



Determine whether the Collaboration Management Strategy and Plan contains approaches and strategies to engage with the right external entities to support the goals of the project in a productive manner.

The updated Collaboration Strategy addresses interactions with both internal and external parties.


Previous

Trend

Verify that the metrics for assessing collaboration partner engagement are appropriate and tracked.

As part of its Collaboration Strategy rollout, the project team established partner collaboration metrics such as participation and level of engagement and is evaluating them on a quarterly basis.


Previous

Trend

G

G

Y

G


Appendix 1. Project Element Value Assessment and Mapping

Value Map Approach

Grant Thornton evaluated the elements of project management outlined within the Florida PALM PMP along with additional critical elements included from our methodology. Because risks and issues in some project categories will have a larger impact on the overall project, we prioritized the project elements based on their relative impact to the project. We evaluated each project management process against the following three categories to determine an overall priority for the Pre-SSI portion of the Florida PALM Project. These priorities are likely to change to some degree once the integrator is chosen. Each Project Management process was weighted either ‘high’, ‘medium’, or ‘low’ for each of the following categories, with the average providing the overall prioritization.

Correlation to Outcomes – how directly related is the project area to desired outcomes of the Project. (e.g. scope management has a high correlation; if desired functionality is removed from scope, it will have a significant impact) Note that this category is weighted twice as strongly in the overall prioritization.

Visibility – how much external attention is this project management area likely to attract

External Impact – is this an area that has direct impact on external stakeholders, or where external stakeholders will have a direct impact on the execution of the project

42


Process Values

The following table displays the project management processes, their relative score in each of the value prioritization categories, and the overall priority value. Going forward, these priority values will be used to weight and prioritize the recommendations identified in the body of this report.

Value Prioritization Category

Project Management Process Co

rre

latio

n to

Ou

tco

me

s

Vis

ibili

ty

Exte

rna

l

Imp

act

Ove

rall

Pri

ori

tiza

tio

n

1. Mandated Criteria

1.1. Scope Management and Change Control HIGH HIGH HIGH HIGH

1.2. Schedule Management HIGH HIGH MEDIUM MEDIUM

1.3. Cost Management HIGH HIGH MEDIUM HIGH

1.4. Risk Management HIGH MEDIUM HIGH HIGH

1.5. Issue Management HIGH MEDIUM LOW MEDIUM

2. Governance

2.1. Project Governance HIGH MEDIUM MEDIUM HIGH

2.2. Decision Management HIGH HIGH HIGH HIGH

3. Project Execution

3.1. Performance Management HIGH MEDIUM LOW MEDIUM

3.2. Staffing Management MEDIUM MEDIUM LOW MEDIUM

3.3. Action Item Management MEDIUM MEDIUM LOW MEDIUM

4. Procurement and Contracts

4.1. Procurement and Vendor Management MEDIUM MEDIUM HIGH MEDIUM

5. Quality Management

5.1. Quality Management MEDIUM HIGH LOW MEDIUM

5.2. Deliverable Management HIGH MEDIUM LOW MEDIUM

5.3. Lessons Learned Management LOW LOW LOW LOW

6. Communications and Outreach

6.1. Communications Management LOW HIGH HIGH HIGH

6.2. Stakeholder Management HIGH HIGH HIGH HIGH

6.3. Content Management LOW LOW LOW LOW

6.4. Collaboration Management MEDIUM LOW LOW MEDIUM


Appendix 2. Data Management Plan Assessment Summary

Summary of Deliverable Review Report

As a part of providing independent project oversight and assessment, Grant Thornton will from time to time review and provide feedback on key project deliverables. One of the deliverables assessed in February 2016 was the Florida Palm Pre-DDI Data Management Plan (DMP), I-SDS2.

Our approach to this assessment was to evaluate the DMP against the criteria laid out in Proviso, the DMP Deliverable Expectation Document (DED) and against best practices, as defined in the Data Management Association’s Data Management Body of Knowledge (DAMA-DMBOK) as well as several general and Florida-specific requirements.

Overall, the DMP addresses each of these areas in a sound manner. The DMP achieves full compliance with the criteria outlined in Proviso. At the time of our review of the DMO, one minor issue existed related to the Deliverable Expectation Document (DED). Another minor issue existed relative to best practices. Also of note, many of the criteria described in the DAMA-DMBOK were not evaluated in this report as they do not apply to a data management plan for a project in the Pre-DDI phase. While not evaluated, these criteria are listed in the report below as ‘not applicable’ because they provide a framework for the evaluation of the DMP as the project progresses into the DDI phase and details about the system and integrator become known.

The following table summarizes any “Red” or “Yellow” findings from the detailed assessment section of the report and maps our associated observations/recommendations to the project health indicator presented in our overall risk reporting:

Summary Finding Indicator Criteria Area Recommendation

DMP01. The DMP has been

reviewed by the project and the Florida Agency for State Technology. Other project stakeholders – such as FFMIS partners or the legislature – have not yet been consulted.

DED Traceability

Before finalizing the DMP for inclusion in the ITN, we recommend that is shared with other key project stakeholders for review and comment.

Y

44


Summary Finding Indicator Criteria Area Recommendation

DMP02. The DMP uses the

DMBOK to outline its approach to numerous data management functions but does not interlink these processes with existing project management structures such as risk management, scope management,

General Requirements

In the section of the DMP covering governance, the document should indicate how the existing project management processes to manage risk, scope, schedule, etc. will support the data governance process.

Updates to Deliverable Review Report

Subsequent to the release of our Deliverable Review Report in February, the project team has acknowledged both of these findings and has taken action to address them as follows:

DMP01: The DMP is scheduled to be reviewed by key stakeholders prior to its inclusion in the ITN, including several FFMIS partners.

Result: The Grant Thornton team will continue to monitor the status of the DMP to ensure that the reviews by key stakeholders are completed as scheduled.

DMP02: Several updates were made to the DMP that addressed the linkages between the DMP and existing project management processes.

Result: The Grant Thornton team feels that this finding has been adequately addressed and can now be closed.

Y


Appendix 3. Interviews and Meetings Summary

The following meetings and interviews were conducted with entities outside of the Florida PALM team during the assessment period covered by this report.

Meeting Type Meeting Summary Agencies

Represented

Planning / working

Project status weekly reviews AST

Agency Agency Interview Department of Citrus

Agency Agency Interview Justice Administrative Commission

Agency Agency Interview State Courts Administrator

Agency Agency Interview Department of Veterans’ Affairs

Agency Agency Interview Department of Juvenile Justice

Status Steering Committee Meeting with legislature and project sponsors

Florida Legislature


Appendix 4. Project Documentation Reviewed

The table below lists some of the project artifacts reviewed by the IV&V team in preparation of this assessment:

Category of Documents Reviewed

Documents Reviewed

Project Reporting Project Weekly Reports

Project Quarterly Reports

Steering Committee Meetings

Project Risks, Action Items, Issues, Decisions and Lessons Learned (RAIDL) Dashboard

SharePoint RAIDL:

Risk Log –risks identified and reviewed

Action Item Log –action items captured and reviewed

Issues Log –issues identified and reviewed

Decisions Log –decisions identified and reviewed

Lessons Learned Log –Lessons captured and reviewed

Project Change Requests Log –PCRs logged and reviewed

FL PALM Document Libraries

Project Standards:

Requirements Review: Library for the requirements review process for DFS Accounting and Auditing, Treasury, and Information Systems.

FL PALM General Site Content

General Site Content:

Action Item Log - Combined action item list for all work tracks including Procurement.

Announcements - List to track upcoming events, status updates or other team news.

Calendar - List to keep informed of meetings, deadlines, and other important events.

Considerations Log

Decision Log - Combined list of BPS, PMO, & Procurement Decision Logs.

Issues Log - Combined list of BPS, PMO, & Procurement Issues Logs

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Action%20Item%20Log/AllItems.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Announcements/Current%20View.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Calendar/calendar.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Considerations%20Log/AllItems.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Decision%20Log/AllItems.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Issues%20Log/AllItems.aspx

47



Documents Reviewed

Lessons Learned - Combined list of BPS, PMO & Procurement Lessons Learned Logs

Meeting Log - Meetings with one or more Sponsors and/or any Non-Project Team members.

Project Change Request Log

Risk Log - Combined list of BPS, PMO and Procurement Risk Logs

Business Process Standardization (BPS) Library Documentation

BPS Documentation:

Schedule – Project Scheduled Baseline and Updates

Status Reports – Weekly Work Track Status & 4 Week Look Ahead Reports to the PMO

Policy Issues – Policy Issue tracking log, DEDs, FLAIR and CMS Reports, etc.

Strategic Plan – BPS 2, 9 and 16 Strategic Plan Deliverables and DEDs.

Level 1 – BPS 4, 9 and 8 Deliverables and DEDs.

Level 2 – BPS 10 and 13 Deliverables and DEDs.

Requirements – BPS 7 Template and BPS 11 and 20

BPS Lists BPS Lists:

Level 1 Requirements List – Business Process Areas Requirements.

Level 2 Topics – Business Process Area Topics.

Level 1 Policy Issues List – Business Process Area Issues.

Project Management Office (PMO) Documents Library

PMO Documents Library:

Project Change Requests

Project Support Tools

Decision Requests

Project Schedule – Baseline Schedule & updated revisions.

Four Week Look-Ahead – Project Schedule 4-week view.

FTE Reports – Staffing

Meeting Documents – Minutes and Agendas

Milestone Report Archive

PMO Contract Management

PMO Forms and Templates

PMO1 Project Status Reports – Deliverable

PMO2 Project Charter revised 02/18/2016– Deliverable

PMO3 Project Management Plan v2.1 – Deliverable

PMO4 Project Management Review – Deliverable

PMO5 Weekly Review of Track Project Status Reports

PMO6 Monthly Project Status Reports – Deliverable

PMO7 SSI Project Management Requirements SOW

Weekly Status Reports

FLAIR Study - Final

FLAIR Study Assumptions and Timeline

Procurement Lists Procurement Lists:

Contract Change Order Log

Procurement Log

Organizational Change Management (OCM) SharePoint Main Page

OCM Main Page:

Track Schedule – OCM MS Project Team Schedule

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Lessons%20Learned/AllItems.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Communications%20Log/CurrentMeetings.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Project%20Change%20Request%20Log/AllItems.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/Lists/Risk%20Log/AllItems.aspx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2F1%20%2D%20Project%20Change%20Requests&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2F2%20%2D%20Project%20Support%20Tools&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2F3%20%2D%20Decision%20Requests&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2F4%20%2D%20Project%20Schedule&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FFour%20Week%20Look%2DAhead&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FFTE%20Reports&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FMeeting%20Documents&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FMilestone%20Report%20Archive&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO%20Contract%20Management&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO%20Forms%20and%20Templates&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO1%20Project%20Status%20Reports&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO4%20Project%20Management%20Review&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO5%20Weekly%20Review%20of%20Track%20Project%20Status%20Reports&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO6%20Monthly%20Project%20Status%20Reports&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FPMO7%20SSI%20Project%20Management%20Requirements%5FSOW&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FPMO%2FShared%20Documents%2FWeekly%20Status%20Reports&FolderCTID=0x012000D8FD8E521DA22A41BAF0EC6930561A22&View=%7bE96A01D5-DCC9-4F54-9DF3-491236638CCE%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/FLAIR%20Study%20-%20Final.docx

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/PMO/Shared%20Documents/FLAIR%20Study%20Assumptions%20and%20Timeline.docx

48



Documents Reviewed

Four Week Look Ahead – 4-week calendar view captured in MS Excel tracking milestones, work products and deliverables.

OCM Libraries OCM Libraries:

Administrative – Folders and documents relating specifically to OCM and administrative business

Ideas – Ideas for Track and project improvement or activities that have not been vetted

OCM Weekly Meetings – Agendas and notes from OCM weekly meetings

Schedule – OCM Track Schedule

Status Reports – Weekly Status Reports

Metrics and Data – Files organized by the event from which the information was gathered including collection method, raw data, data evaluations, and final summary reports

OCM Resources – Papers, presentations, graphics, and other tools and information used by OCM team members to provide service and support to the Project and Tracks (e.g., meeting facilitation tools, mapping exercises, and change management literature)

Project and Track Support – Support documents and work products produced in , organized by Project and Track

Strategies – Strategic OCM documents (e.g., Collaboration Strategy, FFMIS Engagement Strategy, and Website Content Strategy

OCM Communication Tools & Log

Communication Tools

A&A Newsletter – Articles and information submitted to A&A for publication in their newsletter

Project Standard Documents – Proposed and accepted documents and work products related to project standards (e.g., Project Style Guide and Word Template)

Website – Communication planning documents for the Florida PALM webpages

OCM Communications Log – History of project communications captured in MS Excel format.

SDS Documents SDS Documents:

Status Reports – Weekly Project Status Reports.

Strategy – Data Management Strategy, SDS Pre-DDI Track Strategy.

Admin – Archive, RFQ, SOWs, etc.

Requirements – Draft Requirements, Technical Category Ratings, Category Crosswalk, etc.

FLAIR Study Report State of Florida feasibility study regarding the replacement of FLAIR and CMS with a “Commercial off the Shelf” (COTS) ERP solution.

FLAIR Study Cost Model State of Florida cost study regarding the replacement of FLAIR and CMS with a “Commercial off the Shelf” (COTS) ERP solution.

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/SDS/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FSDS%2FShared%20Documents%2F1%2E0%20Status%20Reports&FolderCTID=0x0120009DF715F0990F6744BF2266D65B8CF76C&View=%7bBC34BC5B-99C9-454B-9B54-A1E7980859AD%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/SDS/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FSDS%2FShared%20Documents%2F2%2E0%20Strategy&FolderCTID=0x0120009DF715F0990F6744BF2266D65B8CF76C&View=%7bBC34BC5B-99C9-454B-9B54-A1E7980859AD%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/SDS/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FSDS%2FShared%20Documents%2F3%2E0%20Admin&FolderCTID=0x0120009DF715F0990F6744BF2266D65B8CF76C&View=%7bBC34BC5B-99C9-454B-9B54-A1E7980859AD%7d

http://dfsintranet.fldoi.gov/capitol/osp/pmoffice/FCR-PreDDI/SDS/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2Fcapitol%2Fosp%2Fpmoffice%2FFCR%2DPreDDI%2FSDS%2FShared%20Documents%2FRequirements&FolderCTID=0x0120009DF715F0990F6744BF2266D65B8CF76C&View=%7bBC34BC5B-99C9-454B-9B54-A1E7980859AD%7d

http://www.myfloridacfo.com/Division/AA/140429-DOF05-Final-FLAIR-Study-Combined-v100.pdf

http://www.myfloridacfo.com/Division/AA/140409-DOF05-Final-FLAIR-Study-D5-CH2-Model-v103.pdf

Mirones, Daniel Sent: Friday ...€¦ · 1301 International Parkway, Suite 300 | Fort Lauderdale,...

Documents

Transcript of Mirones, Daniel Sent: Friday ...€¦ · 1301 International Parkway, Suite 300 | Fort Lauderdale,...