Time to think the unthinkable

So the problem as quoted many times before is, "how to demonstrate that
security practices are both working and effective?".

As in most production network and system footprints, the goal posts,
risks and processes are ever moving and changing. How do we measure
security and track a risk management program effectively without making
comparison to some baseline or standard configuration?

The answer is we don't. There isn't any and there never will be for
*your* IT footprint.

So, you ask, what can we measure when trying to prove the unprovable?

The answer lies in making a comparison to something, but to what? Other
organisations perhaps? Too tough it seems, as they don't have nor share
relevant data or reports for obvious reasons. Each IT footprint and
business is built on similar building blocks but is fundamentally
different as the "superorganism" morphs, grows and responds to different
traffic and needs constantly.

What is required is a clone. A sometimes inferior and sometimes superior
clone of the organism. A test subject if you will.

I am not talking about development systems or sub-models of the organism,
but a live fully functional version of the entire entity, which can
assume many states at a whim.

With todays focus on virtualisation and seemingly never ending
processing and storage, surely we could construct a clone of our
business residing all in memory on a single or distributed platform.
This is what we would perform change management upon and measure
against. Think of it, not as a "honey-net", but a "honey-org".

Yeah, yeah, needs X memory and won't be perfect, but better than nothing
and the "honey-org" could be constantly updated from the live management
systems.

With a form of total information awareness we could build a clone of our
systems, nodes and processes - exclusively in software, running on
lower spec hardware perhaps. Here, only time becomes the key factor.

Using something akin to enterprise management systems the information
and images could be gathered to build a clone of our organisation or
enterprise in as much detail as possible which would facilitate a form
of testing that would see us able to demonstrate what would, and does
happen, if certain policies or changes are not enforced.

Three, is the magic number. A production network, a clone network, and
whatever other development or model systems are required.

This would not address physical security in tandem with virtual
security, nor would it be able to fully simulate all users or business
processes, but it is what we are approaching with products like Cisco's new provisioning platforms, Opnet's modeler and risk management via Skybox. The ability to then generate traffic and incidents is also required. SmartBits, IMix, VA and fuzzing till your heart's content including session replay all in one go? Easy huh?

The main problem today is we don't know enough about our environments.
We need to know more. We need to build a fully virtual infrastructure to fully understand our real infrastructure. We may be still quite some
distance away, but in extrapolating the possible futures, this seems to
me to be the only way of demonstrating the "what if" scenarios!

Open up the virtual infrastructure to the net. Offer up your virtual
systems to the bad guys. Transparency is the key both for the bad guys
and the good guys if we really want to make progress. Perhaps.

In theory this is what we pretend to do with "Change Management" today, though the results are more often than not guessed at, divined from experience and faith and are totally subjective.

Yes it is time to slow down the rate of change, make a copy, offer it up
or strip bits away from our clone to see where and how the security
dollars are really being spent. Gimme' a clone, drop it in the lions
den, and I'll give you, at least in part your ROSI(Return on Security
Investment).