The report fails to address the systemic cause of the problems, which is a lack of competent senior management in Department of Defence and the Australian Defence Force. It is not sufficient to have senior staff who are civil service managers and military warriors. The defence force relies on complex systems which require skills in project management, engineering and ICT. The public service and the defence force have personnel who are competent in those disciplines, but too few are being promoted to senior levels. As a result large complex projects are overseen by senior staff who are not competent. More junior staff are well aware of the problems and make recommendations, but these are then not acted on.
One of the other systemic probes with the Collins class submarine is that it was custom built for the Australian Navy. For the follow-on, a proven design should be selected, with the minimum of modifications. The first few boats should built in an overseas shipyard with proven submarine experience. Later vessels can have some local Australian construction.
The report was released in a poorly formatted PDF document, so I have extr5acted the executive summary (minus the figures).
Executive Summary
My Phase 1 Report into the sustainment of the Collins Class was published in December 2011 and put into the public domain. This Report incorporates the work undertaken during Phases 2 and 3. The Phase 2 work was undertaken between March and June 2012 and Phase 3 between September and November 2012.
The Collins Class Sustainment Study has investigated whether the evidence supported the observations made in the Phase 1 Report by the Review Team, which it did. In essence, we set out to answer the following questions:
These questions were tackled using a structured, fact-‐based investigative approach and employed skilled practitioners in the fields of strategy and management, organisation and personnel, business operations, planning and project costing. To complete the work the study team reviewed over 2,500 documents, interviewed over 200 people across the Collins Class Sustainment Program and conducted a Commitment and Culture Survey which 665 leaders and personnel from across the Collins Class Sustainment Program completed. The study teams were supported in their work by the leadership of the four participants involved in the program, the RAN, DMO, ASC and Finance. We recognise that this work could not have been delivered without their committed support to the study. In Phase 3, we considered new reports into international benchmarking of comparator navies, the Collins service life evaluation program and details of initiative programs of the RAN, DMO and ASC. The findings of the study teams are summarised in this report.
- What is wrong now with the Collins fleet sustainment performance?
- What caused the current problems with sustainment performance?
- Will improvement initiatives address these issues?
- What are the recommendations to resolve the remaining issues?
i. What is wrong now with the Collins fleet sustainment performance?
Setting the Requirement
The lack of an unclassified statement of the operational requirement is significant. Although the details are classified and must be protected, a broad unclassified statement of what the RAN requires from the Collins Class would enable a greater understanding of the requirement for all of those involved in the sustainment program. For example, from a fleet of six submarines it is reasonable to expect “two ready to deploy 365 days a year”. It would also be reasonable to expect two to be in deep maintenance at all times. This should be achievable if three submarines are materially available 90% of the time: a level that the Collins Class should be capable of meeting. It is clear from our analysis that this requirement is unlikely to have been met in recent years. This concept is illustrated in Figure 1. Figure omitted Figure 1 -‐ An example of a clear operational objective Note: The RAN, informed by conclusions of the Phase 2 work, has now promulgated an unclassified statement of availability.
Comparison to other Navies’ whole platform availability
To assess the effectiveness of the Collins Class Sustainment Program, it has been compared with data supplied by international navies as reported in the International Benchmarking Report produced by the DMO. Benchmarks of this type have many variables such as differences in program availability definitions, the specific complex features of submarine fleets and their differing demands and operating environments. Nevertheless the differences in availability performance between the comparator navies were sufficiently narrow to develop a reliable benchmark. The comparator submarine fleets used were similar in fleet number and size to the Collins Class. The Collins Class Sustainment Program performance from FY06/07 to FY10/11 is compared with the average performance of each comparator submarine fleet in Figure 2. The analysis indicates the availability performance of the Collins Class has been slightly over half that achieved by the comparable international programs; the time in planned maintenance was about one third greater than other nations; and the maintenance overruns and the percentage days lost due to defects were approximately double that of the comparators. Figure Omitted. Figure 2 -‐ Collins performance compared to other international navies Relative to international comparators
Availability
Collins Class availability compared to the international comparators reduced between FY06/07 and FY09/10, as illustrated Figure 3. The availability reduction is a result of a combination of factors related to legacy design and build issues, reliability problems and the way in which sustainment is organised and managed. There has been an improvement in availability since FY09/10, but the rate of improvement in FY10/11 has not been maintained subsequently. The program appears to have stabilised, but this is too early to confirm. The level of performance at 56% of the international comparators was significantly below the RAN’s requirement for FY11/12. Figure omitted. Figure 3 -‐ Comparative Availability Low level materiel availability is driven by three key factors:
The relative contribution (opportunities) of these factors to improve availability performance to match the Material Ready Days (MRD) benchmark is shown in Figure 4. Figure omitted Figure 4 -‐ Opportunities to improve availability to MRD benchmark This data indicates that fundamentally changing the usage-‐upkeep cycle and shortening the maintenance periods, and managing them in a way that reduces time overruns will yield the biggest contribution to improving the number of MRDs achieved. The MRDs due to defects cannot in practice be eliminated altogether. All nations lose some MRDs to defects. However, when operational days are lost they have a far more disruptive effect than days lost in maintenance and it is still very important to improve reliability and reduce the time taken to repair defects when they occur.
- Long planned maintenance periods;
- Overruns to planned maintenance periods;
- and Defects to the submarines when they are outside of maintenance periods (poor reliability), which can be exacerbated by inadequate availability of spares.
Reliability
Figure 5 illustrates the reliability of the Collins Class through the Priority 1 (P1) urgent defects (URDEFs) reported each year normalised by the average number of submarines in-service (not in planned maintenance) in each year and referenced to FY06/07. Whilst the number of submarines in-‐service reduced in FY08/09 and FY09/10, it demonstrates that the number of defects per submarine increased, greatest of which was for Priority 2 (P2) URDEFs. The defect rate improved in FY10/11, which aligned with the improvement in availability noted above. We did not analyse FY11/12 as the data is a mixture of planned and actual. The URDEF rate in FY10/11 was significantly above levels achieved in earlier years (FY06/07 and 07/08). The overall upward trend is indicative of poor reliability and obsolescence management in earlier years. The original design shortcomings should have been removed over time through sound application of these basic sustainment management processes. Figure omitted. Figure 5 -‐ URDEFs normalised by the average submarines in-‐service (relative to FY 06/07 P1 URDEFs)
Cost
The cost of the Collins Class Sustainment Program was generally stable from FY06/07 to FY09/10 with cost increases in FY10/11 for the purchase of additional spares and the implementation of an improved maintenance program. The year FY06/07 is used as the datum year because we have more confidence in the availability data from that year forward. It does not represent a datum point for the cost being adequate to fully fund the Collins Class Sustainment Program. As illustrated in Figure 6, cost effectiveness can be measured by comparing the total cost versus availability achieved ($m/MRD), this is indicated by the green line on the graph, which illustrates cost effectiveness declined from FY07/08 to FY09/10. Additionally, cost effectiveness improvements commencing in FY10/11 coincide with increased investment in the program, however the improvement and the investment cannot be directly linked in time. Benefits from consuming additional resources may not be visible for 18 to 30 months after committing to the investment as the lead time required to procure and deliver spares can be one to two years and the time required to improve maintenance schedules that deliver performance may be of several years duration. Figure omitted. Figure 6 -‐ Relationship between sustainment cost and availability (MRD) Figure 7 shows a comparison of three measures between Collins and benchmark data indicating the rate of consumption of man-‐hours during comparable Full Cycle Dockings, the effective man-‐hour rate and the cost per MRDs achieved. However, international comparator data cannot be considered as precisely comparable to the data relating to the Collins Class given the wide variance in accounting practices and attribution of costs to the various activities. Nevertheless averaging the three measures provides an indication, not an absolute, that the cost effectiveness (defined as the cost per MRD) of the international comparators is at least twice that of the Collins Class Sustainment Program. This analysis includes all the costs across the entire Collins Class Sustainment Program and not just the cost of the maintenance activity or the ASC contract. Therefore improving cost effectiveness will require improvements across all the participants in the Collins Class Sustainment Program. The cost effectiveness would more closely match the international navies if the availability was increased to benchmark. Figure omitted. Figure 7 -‐ Cost effectiveness comparison to international program Relative to mean of comparators
Organisation Implications of the In-‐Service Support Contract
The In-‐Service Support Contract (ISSC) places responsibility on the ASC for delivering outputs and uses Key Performance Indicators (KPIs) related to these outputs to measure and reward the performance of the ASC. This move to performance based contracting will significantly change the current roles and responsibilities of the DMO, ASC and the RAN. Our high level analysis of the Collins sustainability responsibilities (the Value Chain) shows that currently the DMO is involved in eight of the activities, the RAN six, ASC nine and the Capability Development Group two. Under the In-‐Service Support Contract, using “good practice” guidelines, these responsibilities should be realigned such that the DMO is involved in four activities, the RAN eight, the ASC eleven and the Capability Development Group two. In order to deliver the benefit expected from the ISSC it is essential that the necessary organisational adjustments are made to align resources with the good practice value chain.
ii. What caused the current problems with sustainment performance?
Our analysis has identified 20 key issues we believe are driving the low level of sustainment performance and we traced these back to five root causes which are equally significant:
- Unclear requirements – Operational requirements were not defined in a way that can be effectively translated to sustainment activities. There is a need for a clear unclassified top level requirement for the Collins Class program that can be used to drive the sustainment program through effective planning and contractual incentives for all participants. The RAN, informed by conclusions of the Phase 2 work, has now promulgated an unclassified statement of the required availability.
- Lack of a performance based ethos – As detailed in the Phase 1 Report, in the past there has been a clear lack of a performance based culture across the Sustainment Program. However, there have been significant improvements over the course of this study.
We welcome these initiatives, however they need to be coordinated across the Collins Class Sustainment Program to most efficiently achieve the availability outcomes.
- The development and transition to the ISSC (between DMO and ASC) is recognised as a significant step forward in the implementation of a performance based culture. This should support a change in ethos, as the historical contractual arrangements between DMO and ASC were not performance based.
- Although defined KPIs existed in prior versions of the Materiel Sustainment Agreement (between RAN and DMO), it was unclear how the RAN held DMO to account for achievement against these KPIs as targets were not apparent. The introduction of the new Materiel Sustainment Agreement between the RAN and DMO now contains clear performance requirements and is a similarly positive development. The RAN has a significant role to play in onboard maintenance, which is now clearly articulated in the new Materiel Sustainment Agreement.
- As the shareholder of the ASC, Finance has reviewed the existing performance arrangements and is introducing a new comprehensive performance management framework for the business.
- These new performance targets linked to the new top level requirements are essential to support a performance based ethos.
- Unclear lines of responsibility – Many key roles and responsibilities at all levels within the Collins Class Sustainment Program are not clearly defined or understood from an organisational and an individual perspective. This results in blurred accountability where duplications and gaps in responsibility have developed over time. This is particularly evident between DMO and ASC. Additionally the customer-‐supplier relationship between the RAN and DMO has not been sufficiently clear. The new Materiel Sustainment Agreement for 2012-‐2014 has now established the clear responsibility on the DMO to deliver materiel sustainment for a given budget with a corresponding obligation on the RAN to supply crews and on-‐board maintenance to support the program.
- Poor planning – The lack of a clearly stated long-‐term strategic plan prevents accurate lower level plans and targets being established and achieved. This results in unclear and conflicting requirements across the Collins Class Sustainment Program leading to limited mechanisms with which to drive a unified management approach. A lack of planning at the strategic level means lower level plans for maintenance and military operations are built in isolation and are not managed in a portfolio manner.
- The absence of application of an asset management strategy has also resulted in poor obsolescence and a lack of reliability management. This has created a ‘bow wave’ of reliability related defects and obsolescence, resulting in an uncertain and uncoordinated planning baseline. This is regarded as a key factor that is impacting performance of the Collins Class Sustainment Program in delivering sustainment activities.
- Lack of a single set of accurate information to inform decision making – Effective systems and processes in addition to accurate and timely data are crucial to achieving an informed position upon which organisations can make decisions. The Collins Class Sustainment Program is not in a position, from an information perspective, to make optimised long-‐term decisions. There are multiple systems and datasets in use for financial, maintenance and supply chain activities. In many cases these are not linked, resulting in data integrity issues. The lack of a “single version of the truth” means decisions are unlikely to be consistent or accurate.
iii. Will current improvement initiatives address these issues?
The current initiatives underway in the RAN, DMO, and ASC align with the majority of our findings. Many of the current initiatives are aimed at addressing the fundamental sustainment management issues of maintenance obsolescence and reliability management, supply chain improvements, establishing adequate crewing, and introducing performance based management. However, they will not resolve all of the issues or root causes identified in this report. In the main the existing initiatives are aimed predominantly at resolving specific sustainment issues. There are some key issues that are not being addressed at all, mainly at a strategic level across the Collins Class Sustainment Program, while other issues are being addressed by multiple initiatives. These overlaps and gaps will require coordination and realignment of their ownership. Where key issues are not fully addressed the current initiatives will require enhancement. The Commonwealth has embarked upon a Submarine Life Evaluation Program study for the Collins Class. The key issues identified in the report are synergistic with this study. These were: obsolescence and obsolescence management, design and the associated growth margins and their consumption. This program should it be implemented, while it is not an availability improvement initiative, is essentially extending the sustainment boundary and will magnify many of the sustainment issues we have identified.
iv. What are the recommendations to resolve the remaining issues?
Based on the experience of the Review Team, the following recommendations are made and should be implemented to enable meaningful change to occur. They are set against each of the five root causes as detailed in Table 1. The RAN, DMO, and ASC have already implemented or are addressing some of these recommendations. Progress is detailed in the body of this report.
Table 1 -‐ Recommendations
Root cause
Recommendation
Unclear requirement
R1. Set a realistic target for the DMO to deliver MRDs and incorporate in the MSA
R2. Define a clear (unclassified) requirement for the sustainment program Lack of a performance based ethos
R3. Implement the ISSC to encourage performance based behaviour
R4. Finance to strengthen and broaden the accountability framework for the oversight of ASC
R5. Strengthen the RAN as the Intelligent Customer for Sustainment
R6. A forum to bring together all suppliers within the Collins Class Sustainment Program R7. Coordinate existing initiatives, accept recommendations from the Phase 3 Report and coordinate implementation according to the Implementation Strategy
R8. Develop and implement a contracting strategy
R9. Create a collaborative framework known as the ‘Enterprise’ without diluting the individual responsibilities of the participants R10. Improve leadership skills, knowledge and experience
Poor planning
R11. Defer HMAS COLLINS Full Cycle Docking (FCD) and improve maintenance planning
R12. Develop an Asset Management Strategy for sustainment
R13. Availability requirements in the MSA should be derived from the IMS and a working level plan generated
R14. Develop a through-‐life capability management plan reflecting the updated requirement
R15. Define and endorse an Asset Management Plan
R16. Implement and complete a fully-‐integrated sourcing and materials supply support program under the ISSC
Unclear lines of responsibility
R17. Treat defects occurring prior to the completion of Sea Acceptance Trials (SATs) as part of the contracted maintenance period R18. Review and where necessary improve procedures to audit O-‐level maintenance and records
R19. Create a Head of the Submarine Profession
R20. Develop a clear line of authority for maintenance of the design intent R21. Develop and implement a workforce strategy to specifically address skills shortages at the management level
R22. Develop and implement a plan to resolve loss of Naval Engineering Skills
Lack of a single set of accurate information to inform decision making
R23. Improve adequacy of the Ships Information System and implement the use of onboard portable technology to aid in maintenance efficiency R24. Develop Enterprise-‐wide IT strategy and information management strategy R25. Develop cost baseline / model and supporting processes for sustainment program...
From: "Study Into the Business of Sustaining Australia’s Strategic Collins Class Submarine Capability" by Mr John Coles, Paul Greenfield and Arthur Fisher, November 2012
No comments:
Post a Comment