There are many good reasons to run a Proof of Concept (PoC) before deploying new technology – starting with reducing project risk and gaining internal confidence that the chosen solution will address your use case.
But leading a successful cyber security solution PoC takes a laser-like focus on project management (aka scoping, scheduling and herding cats), along with softer skills like relationship building, negotiation and patience. Following industry best-practices doesn’t hurt either.
I’ve been involved in multiple critical infrastructure PoCs – from firewall to network monitoring implementations, and have often been asked about the most important considerations when planning for a successful Proof of Concept. Read on to learn what I think matters the most.
Why Run an ICS Security Proof of Concept?
As I mentioned above, a PoC can improve the likelihood of a tech project’s success. That’s partly because it helps you develop a deeper understanding of your needs. Is visibility into your ICS network, or knowing where your vulnerabilities lie more important? Or is it more about defending your operation against malicious cyber threats?
A PoC forces you to be specific about requirements, and helps you build a project value proposition that can be socialized for approval by the executive team. It also helps align departments like IT and OT towards a common goal. Beyond that, a PoC also proves (or not) that the proposed solution will meet your needs, and increases confidence that the selected technology is right for your organization.
Five Things to Consider When Planning Your ICS Security Proof of Concept
Okay, enough with the basics – you’re likely anxious to get your PoC started. So here are five things that I think are really important to consider when planning a Proof of Concept
- Know your problem statement and use cases inside out – Typical PoC deliverables include a problem statement, use cases, project scope and plan, success criteria, an evaluation (scoring) model, and a project schedule.Use cases and a concise problem statement help ensure the project and PoC stays on track. Problem statements define the need from the 30,000-foot level. Use cases reveal specific applications of the problem and outline how a potential solution will meet the need, or problem statement. Without a problem statement and at least one use case identified up front, defining success criteria becomes ad-hoc, increasing the likelihood that the PoC will fail.
- Focus on objective metrics and use subjective metrics wisely – Must-have, objective metrics that match your specific operating environment are generally black and white. The vendor solution integrates with FireEye’s SIEM, or it doesn’t. It offers passive and active network monitoring, or it doesn’t.It supports the Siemens S7 ICS protocol, or it doesn’t.But about fifteen percent of my PoC evaluation model involves subjective metrics. For instance, when running a PoC with network monitoring solution providers, my team selected a success criterion based on the “culture” of the company we would be partnering with. Of course, the vendor needed to meet our technical requirements, but we also wanted to make sure that the company selected would be great to work with. Particularly if we needed to solve an unexpected challenge, or had to think outside the box to stay competitive through innovation in the future.Is there an objective metric that determines the culture of a company? Not really. Relationships are built on interaction with company representatives and developed over time. Yet, this concept became a significantly weighted criterion in our selection process.The continuous monitoring market is relatively young, and will experience significant disruption over the next few years. In a case like this, it was clear that I needed to work with a company that shared similar values and exhibited an earnest interest in working with me over years of growth and change.
- Handling scope creep – Scope creep will occur no matter how hard you try to contain it. My advice is to anticipate it, prepare for it, and decide early on how you are going to deal with it. When working with vendors, new ideas or inspirations may come to light – and can actually help define the best solution. This is good scope creep. Other times, ideas distract from the core of the problem statement and become undesired scope creep. Have a mechanism in your PoC framework to evaluate scope creep and determine if each case should be included or excluded.
- Testing environment – With critical Infrastructure operations like oil & gas, electricity and water, it’s rare that a PoC would be performed in the production environment. So, success of the project will hinge on how closely the lab mimics your production environment. Ensure the test lab uses the same infrastructure with the same configuration and firmware as your production environment.
- Executive buy in – Planning a PoC starts with identifying a Champion – someone with a deep understanding of the problem, and the environment and conditions you operate within. This person plays a critical role in making sure the project moves forward by getting the right stakeholders /sponsor and team members on board. They also need to work fast to remove roadblocks and ensure that deadlines are met.But whether the project is driven by management (top-down) or grass-roots personnel (bottom-up), obtaining executive buy-in is clearly a critical step in the process. Someone from the executive team should be defined as a Sponsor. This person needs to be fully committed to finding a solution to your problem statement and be willing to assign or budget funding for the selected solution.If yours is a bottom-up project, you have some extra work to do. Obtaining buy-in of your project, problem statement and proposed solution will require patience to gain executive traction, but your PoC will be dead in the water without it.
Nozomi Networks currently has more than 100 Proof of Concept (PoC) installations underway, and if the past is any indication, nearly all of them will quickly transition to working deployments. With that much experience running well-oiled operational visibility and real-time OT cyber security PoCs, you’re pretty much guaranteed to be a Champion for your organization.
If you’d like to explore launching an ICS security PoC for your organization, the Nozomi Networks team is here to help. To get started, simply contact us.