A recent article Interesting Times for Distributed DataCentres by Paul Strong (Ebay - Distinguished Research Scientist ) makes a number of interesting points:
- For Web2.0 Services to scale, you MUST back-end these onto massively horizontally scaled processing environments.
- Most Enterprise datacentre environments are moving towards, or could be considered as, priomordial Grid type architectures.
- What is really missing is the Data Centre MetaOperating System - to provide the resource scheduling and management functions required.
Whilst VC and major Systems Vendors are happly throwing money into expounding the virtues of loosely coupled business models enabled by Web2.0 and all things WS-SOA; somewhat perplexingly, they also continue to invest in managment / virtualization / infrastructure solutions which drive tight couplings through the infrastructure stack. Examples include data centre "virtualization" or, as per my previous blog entry on the Complexity Crisis, configuration / deployment management tools.
Hence, industry investment seems to continue to favor the technology equivalent of the "command economy" in which the next generation of distributed Grid data centre is really just one more iteration on today's; central IT organisation control/manage and allocate IT resource in a rigid hierarchical/control command structure. The whole environment is viewed as rigid system which one centrally controls at each layer of the ISO stack; approaches that continue the futile attempt to make distributed environments behave like MainFrames!
What is actually needed is a good dose of Free Market Economics!
- Business Services dynamically compete for available resources at each point in time,
- Resources may come and go - as they feel fit!
- Infrastructure and Systems look after their own interests, and optimise their behaviors to ensure overall efficency within the Business Ecosystem.
You simply cannot beat an efficient Market!