How do you practise for a cyber incident?

Constituents
This publication is for you if you are responsible for your organisation’s digital resilience and want to know what exercising can do for you and how to take the first steps.  
 

Background: What is exercising?

Exercising is putting into practice what you have learned in a controlled and safe environment. You and your employees will build experience, which will allow all of you to take more effective action in the event of a cyber incident. Regular practice ensures that back-ups can be restored quickly after a ransomware attack, for instance, that you communicate effectively during a data leak, and that you don’t make mistakes when managing a cyber crisis. There are several exercise methods, including: 

A tabletop exercise

A discussion-based exercise where employees run through a scenario together and practice their roles through a structured discussion. In a tabletop exercise, the exercise leader presents elements of a scenario in stages, challenging participants to pick up their roles. An example is a fictitious crisis where the scenario describes the crisis and the information available to the crisis team. After the tabletop exercise, participants are familiar with their roles during a crisis. They know with whom they need to communicate and who will make decisions. 

Targeted/small-scale practical exercise

Where a specific component of the cybersecurity or incident response processes is tested. Examples are a technical team restoring back-ups or employees reporting phishing mails. 

A cross-organisational exercise

Where multiple organisations jointly engage in a simulated (large-scale) cyber incident. Such exercises may take several days and aim to practice alignment between chain partners in an (inter)national cyber crisis, for instance. Examples of this type of cross-organisational exercise are the Overheidsbrede Cyberoefening (government-wide cyber exercise), ISIDOOR and Cyber Europe.

Exercising and testing

The terms ‘exercising’ and ‘testing’ are sometimes used interchangeably as synonyms. While there is some overlap, there is a difference in focus. Exercising is (virtually always deliberately) about practicing (security) measures to improve the organisation’s experience whereas testing is about assessing the effectiveness, efficiency and quality of security measures that have been implemented. You can look at exercising as doing a trial examination, where testing is the examination itself. 

In a wider sense, we distinguish four other steps in addition to exercising and testing to learn or improve skills for personnel and the organisation: educate, train, exercise, test, assess and learn (in Dutch this is often abbreviated to OTOTEL). 

Education and training is where employees acquire new skills. Exercising and testing are focused on improving experience of and putting these new skills to the test. Assess and learn are about identifying and adopting issues requiring improvement (that emerged during an exercise/test). This document focuses primarily on ‘exercising’ but you should always look at exercising in a broader perspective.

Step 1. Outline an exercise plan

When you are just getting started with exercising, your organisation may have to get used to it. Perhaps you have a small budget to start with and you still need to prove that exercises are useful. This is not necessarily a problem since you can also conduct effective exercises with limited means. It is important, however, that you prioritise and align your exercises with the needs, objectives and capabilities of your organisation. 

You need to understand the objective and scope of your (first) exercise to begin with. Use the questions below to get an idea of what these are. It helps to discuss these questions with your main stakeholders, for instance the owners of management processes, and summarise them in a (strategic) exercise plan. 

What do I want to practice (first)? 

Exercising is most effective if it targets activities with which your organisation has limited experience. What is most effective for your organisation? This could include the following:

• The (most important) risks to your organisations and the associated mitigating measures. 
• A list of incidents and the activities executed in the course of the incident. If you execute an activity frequently in actual practice already, it does not need extensive exercising.
• An overview of past exercises and the exercise results and points requiring improvements. 
• Recent changes in the processes that are relevant to your digital resilience and that you want to put into practice in an exercise.

What do I want to achieve with an exercise? 

There are several reasons for exercising. Think about the learning objectives that you want to realise beforehand, such as:

• Improving employee experience. Having policies and procedures is one thing but knowing how to execute them effectively in an incident is something else entirely. It is only through exercising that your personnel will be able to take effective action. Especially with an eye to (natural) employee turnover, regular exercise is a good way to ensure effective and focused action.
• Role awareness. Your organisation’s employees may underestimate cyber threats or assume that security is primarily a task for the IT department and at least not their own responsibility. Exercises show employees what their role is during a security incident and why. 
• Improving collaboration between teams and chain partners. Digital resilience is nearly always a team effort and, in today’s intermeshed digital landscape, often cross-organisational as well. Doing a joint exercise gives you the opportunity to work on effective collaboration.
• Required improvements and refining measures. Exercising enables you to reveal weaknesses in your digital resilience. Perhaps you will find that certain processes and protocols are not executed correctly, such as yourincident response plan, or communication between departments and external partners is inadequate in a crisis situation. These insights will allow you to implement improvements.
• Executive awareness. A crisis exercise is a perfect opportunity to involve the management board in your digital resilience. In a crisis exercise, the board has an important decision-making role and is actively challenged by the risks resulting from a cyber attack. This improves your board’s awareness of its role and the need for suitable digital awareness.
• Compliance. Exercise is vital to comply with laws and regulations and for security standard certification. The GDPR, Cbw (Dutch Cybersecurity Act) and ISO 27001 all set requirements with regard to incident preparedness and security measure testing, for instance. Through regular exercises and documentation of findings, you can demonstrate the effectiveness of your incident response procedures. 

Who need to be involved? 

You can only achieve your learning objectives when you engage your relevant stakeholders. Create a list of all relevant stakeholders and involve them in designing and executing the exercise.

Is my organisation ‘prepared’ for exercise? 

Exercises are most effective when participants have already been trained in their roles. If employees are not adequately aware of their roles and what they are expected to do, you should focus on training and education first.

What capacity and capabilities do I have and need? 

A simple tabletop exercise focusing on basic crisis communication requires less capacity and fewer capabilities from your organisation than a multi-day, cross-organisational simulation. Be aware of what you can ask from your organisation to outline a suitable exercise type. This involves not just money and resources but also the time you are asking participants and facilitating personnel to invest.

What risks do I accept? 

Every activity, including exercises, may come with risks. Different types of exercise are associated with different risks. A tabletop-based exercise does not represent any significant risk to your business operations. ‘Live’ switching to emergency power may shut down operations if something doesn’t go as planned. It is important to map these risks in consultation with the relevant stakeholders and describe how they can be mitigated. Ask risk owners to approve the exercise.

Step 2. Develop and execute the exercise

Once you have outlined the exercise, you can develop it and put it into practice. Take the following good practices into consideration.

Seek out inspiration

Every exercise has its own specific context and content, but it will never be completely unique. We recommend not reinventing the wheel. Leverage existing exercise types, frameworks and scenarios. If you are starting out with exercises, use (simple) tabletop exercises to design group exercises that focus on process and communication, and a small-scale exercise to practice more technical elements. 

Learn from peers and chain partners

It is possible that organisations and businesses in your industry (or chain) already engage in exercises on a broader scale. Examples of this type of cross-organisational exercise are the Overheidsbrede Cyberoefening (government-wide cyber exercise), ISIDOOR and Cyber Europe. Consider joining such exercises to find inspiration. You may not have to participate in an exercise directly, perhaps you can be an observer first. This will allow you to learn without big investments in time, money and resources. 

Design a realistic scenario, including time pressure

Credibility and relevance help the participant. Let everyone take on their real role on the basis of their expertise. Under higher pressure, you will find that some employees cannot work with, and deviate from, the regular process for instance. If a scenario does not feel ‘real’ to the participants, show that you understand them but ask them to work with the scenario. Make sure to include their findings in the review, however.

Start small – but get started 

Your exercise needs may be broad and varied. It is good practice to exercise with a smaller scope, particularly if your organisation does not have a lot of experience with exercises. If you want to get started with crisis management exercises, for instance, do not begin with a cross-organisational simulation but go through a fictitious, well-structured scenario in a tabletop exercise. Starting small allows you to build exercise experience, and enables you to demonstrate how exercising adds value while not requiring a lot of capacity. 

Make clear that this is an exercise

While a test may be unexpected in nature, it is generally better to announce an exercise well ahead of time. Communicate that people are allowed to make mistakes during the exercise (you can learn from them!) and there will be no negative consequences. This contributes to the safe learning environment you need for a successful exercise. You should also clarify that this is an exercise on all of the documents you use for practice, such as a scenario description or flip chart, to avoid it being used out of context.

Division of roles for the exercise

Exercise participants must be able to focus entirely on the exercise. If they want to say something that is not a part of the exercise, let them communicate it explicitly. Make sure that supporting roles, including that of exercise leader and observer, are not part of the actual exercise. Value good support: exercises that are not managed effectively, will have less value for the organisation. 

Record choices, issues and points requiring improvement

Ask an observer to record how the exercise is executed. Record what key decisions are made as well as any uncertainties that arose in the course of the exercise. These observations are important to the review and follow-up. Also keep the learning objectives in mind. 

Hiring expertise

Designing and guiding a successful exercise requires expertise and capacity. If these elements are not (immediately) available within your organisation, you should consider whether it is wiser for your organisation to develop its own expertise and/or call on a commercial party:

• You can develop expertise within your own organisation by asking an employee to participate in a course or workshop in organising and executing exercises, for instance. Or participate in exercises at peer organisations to exchange knowledge and experience
• There are (commercial) providers that specialise in training and organising exercises. An external provider may also be the solution if you do have the expertise but lack the capacity to prepare a large-scale exercise. You should of course make sure that the provider can deliver what you need on the basis of your strategic exercise plan.

Step 3. Follow-up after the exercise

After the exercise, take time to think and talk about what went well and what could be improved next time. This is a great way to conclude the exercise and allow participants (and yourself as the organiser) to receive the input you all need to learn. We suggest the following actions:

Review the exercise

While practice makes perfect, no exercise ever is. Give participants space to reflect on the exercise itself. What was well organised about it and what should be improved? Did the participants achieve what they hoped for? And did it lead to the intended learning objective? At the end of the exercise, allow time to collect feedback in a group setting. Participants should also be encouraged to share additional reflections in the days after the exercise.

Create an improvement plan with concrete actions

The findings and experiences will reveal shortcomings in your (management) processes and measures. Which processes require additional attention? For instance, is there inadequate communication between different teams during incident response? Are there specific technical measures that need to be configured more effectively, or are there gaps in employee training and education? 

Discuss these findings with participants when closing the exercise. You should also discuss the findings with the relevant stakeholders after the exercise and jointly formulate an improvement plan. Prioritise the improvement actions and ensure that key (and cross-departmental) improvement actions are included in management reports so the board and upper management remain involved. 

Turn exercising into a cyclical process 

Exercising is an activity that makes a proven contribution to your organisation’s digital resilience. It should be seen as more than a one-off activity; it is a cyclical process in which you work on improving your cyber resilience in stages. Use the tools listed in the section ‘Outline an exercise plan’ to do this as effectively as possible so you can start building more and more on previous exercise experience. You may also want to establish an exercise schedule to plan some elements, such as an annual crisis exercise, at regular times.

This publication was created with contributions from
the Cyberchain Resilience Consortium (CCRC), De Nederlandse Bank (DNB), TCT Rijk, Ministry of Education, Culture and Science, Eye security, Esoxit, Het Normo (TCT Rijk and Het Normo).