Sunday, August 21, 2005

S.O.E. ( Standard Operating Environment )

Well, even if you use NetFlow on routers / switches why not include something like Argus [ http://www.qosient.com/argus/index.htm ] in all your standard host builds limited to its own slice / filesystem ( or implement some log rotation.. ) so the system or host itself builds a historical log of network relationships for troubleshooting, forensics etc etc

Sunday, August 07, 2005

How do we know about History? What are we doing wrong today....

Hmmmm.. simple premise.... we only uncovered much of what we know today about previous civilisations due to the mark they made upon the world, whether the information was intentionally created for recording purposes or that which was an unintentional byproduct of something else they did / used or created.

Here, the concepts of the intentional lifetime of data and the medium of storage chosen are of utmost importance. ( additionally data format / language and physical / logical interface to the data are of concern )

Some remnants of a society such as architecture may be considered a byproduct, however many buildings such as the pyramids of Egypt and South America were built to last the ages and were intended to be a legacy of the then rulers or of the civilisation itself. Funny that in the current modern era, we have sprawling metropolis' of concrete and steel which will in theory also last the test of time, but we don't in essence continue to write or record anything on mediums with similar longevity. Cave paintings and vellum scrolls when in the right conditions can last for thousands of years and convey stories and records of life as it was, and lessons for future generations whether intentional or not. Imagine if you will if the Rosetta Stone was written on a sheet of modern paper, saved on a harddrive in a proprietary format or burnt to a CD-R or DVD...... how long would it last, and what would future generations lose out on or be deprived of?

Maybe you are starting to see my point? We have seen amazing advances in the current and last century, mainly attributable to the rapid increase and spread of information. Cumulative knowledge allows for rapid progress. More raw human processing power, if you will, all connected and digesting reams of information, making inferences, connections, theories, statistical observations and basically learning, refining and increasing the sum knowlege of all humankind. Hopefully making things better and not worse!

Now imagine .... we destroy ourselves in a nuclear holocaust accompanied by a huge EM pulse ( electromagnetic pulse ) that wipes out most, but not all of the digital data on the planet. The end of the current global age of the internet and digital data. We need to start from scratch but most of the engineers and basic information for building complex circuits and the means to access any of the remaining survivable information is gone to us? Operating systems, source code and close to all raw data would be gone.

One could well ask if it ever did then actually exist? How much are we missing from that which was daily life for the Egyptians, Greeks, Romans, Incas etc etc... From what we have found thus far.. e.g ruins, artifacts, personal effects, architecture, farming and certain amounts of business and governmental records of the time - we build a picture of the politics, philosophy, medicine, science, mathamatics, law, ethics etc of these people's lives and overall civilisations....

Many of these civilisations either fell, mutated, diverged or imploded.... again imagine if you will the sum of all human knowledge available to us should we have had a cumulative repository over the past few thousand years... maybe we would be more advanced or maybe we would have wiped ourselves out properly, once and for all!!!!

We are creating and learning at a pace never seen before in human history ( well to the best of our current knowledge based upon what we have found.... how would we really know unless they wrote it down or *all* the information and records had survived? ) what happens when our civilisation comes to an abrupt or bitter end?

Should we be doing more to ensure the information we create and learn about ourselves, our civilisation and our environment is given the intrinsic longevity it deserves.. if not for our children, for future races of humans, for the historians, researchers, teachers and perhaps those that once again one day try to rebuild society out of the dark ages?

Aside: This question also begs an answer to the issue of complexity in many of our sciences and systems and how we represent them. Ask yourself if it would be easy to rebuild, reproduce or look at a current high level system, grasp the underlying concepts and reproduce the outcomes. These systems I speak of could be anything from computer systems, to law systems to social systems. We are building a house of cards with no thought for my favourite question, not "Why" but "What if?".....

Aside II: Do we really actually care about the "What if?" or would it stifle our creativity and the speed of advancements if we were to spend more time ensuring the integrity and longevity of our cumulative knowledge? Why are we rushing so far and fast ahead in to the unknown, we'll still get there eventually... it will still be unknown a week from next Tuesday! Maybe it's time to slow down and take a timeout, have a 'kit-kat' and then take a really good long hard look at what we as a race are really doing and trying to achieve... Also is it sustainable and recreatable should we break it? The wonderful concept not of "How well does it work?", but "How well does it break?" comes to mind...

Right now I see a huge risk to society at large.

"How we represent, store, archive and share digital information. "

I believe the sum of all human knowledge is in danger... let's at least start by ensuring we could start from scratch again. ( or someone or something else could.. what would an alien archaelogist make of all this should they visit earth after we have reduced ourselves to another pre-industrial age again? "Bloody amateurs!" )... oh and here's an interesting byproduct.... maybe a new paradigm would create a template for human learning that creates a code or method for how to educate children so they do not need to spend X times as long comprehending a mish-mash of overlapping disciplines by starting from scratch each time they enter a new field of study. Some may agrue that that is the role language fulfils, a symbolic representation of ideas and concepts to allow them to be expressed and communicated.

There is no handbook for parents, there is no common teaching system other than repetetively hammering information in to childrens skulls. The advances we could make as a race if we harnassed and guided the abstract thought processes of children. ( possible focus "edutainment" )

Libraries and museums perhaps need a bit of a rethink and some real funding?

If we want to build a new society the 'Tipping Point' will start with the children.
If we want to 'keep' and progress our society we need to focus on 'keeping' the cumulative information safe and healthy.
If we want to advance our society we need to eliminate fear, greed and inequality.

Hmmm... rant over.. time to watch the Simpsons....

Some fun links to projects / papers:

Information Longevity http://sunsite.berkeley.edu/Longevity/
NARA National Archives and Records Administration ( American, http://www.archives.gov/ ) ERA ( Electronics Records Archive ) http://www.archives.gov/era/index.html
OSTA Optical Storage Technology Association ( http://www.osta.org/ )