Massive online databases vs. individually siloed data…lets take a look.

There is a movement going on, right now.  Companies are starting to abandon large subscription databases and building their own silos of data.  Why?

Lets first examine the general trend of technology.  When a new technology first gets introduced, it tends to be (1) more complex, (2) more expensive and (3) centralized.  Job Boards for example. First there were the large boards like headhunder and Monster.com, next niche job boards, and then large corporate job boards.  Now even small recruiting firms post their own job postings on their own web sites.   The trend once a technology matures is (1) less complex (2) less expensive and (3) ubiquitous and decentralized.  Less complex because the technology is streamlined and reengineered and less expensive and decentralized due to technology improvements and economies of scale.

Watching this trend has been one of my hobbies, it’s universal like the 80-20 rule.  It’s time to give it a name.  Expensive-Niche-Decentralized or E-N-D. 

An entire series of technologies that follow the END trend.  Web based CRM is also starting to follow this curve.   Salesforce.com = the early days of monster.com.  In the last few years, many new CRM’s specific to vertical markets have sprung up.  Recently there is a movement to self-hosted web based CRM.  SugarCRM is open source and Microsoft CRM can be hosted in-house.   CRM, even web-based, is decentralizing.

Ignoring this trend is eqivilant to putting your head in the sand regarding Moore’s law, Kryder’s law, or Nielsen’s law.

What are the variables that will cause data siloing to follow END?

Read the rest of this entry »