The Single Point of Truth: Broadlook Technologies Takes a Big Move Towards the Cloud

"With CRM Shield, we are solving the actual disease of dirty data" Donato Diorio

Donato Diorio

Welcome to our continuing coverage from HR Tech in Chicago. Donato Diorio is a pioneer in the field of Internet research. As software architect and the owner of a top billing placement firm, Donato envisioned applications that could automate many of the most time-consuming research functions performed by his recruiters. With the assistance of a team of developers, Donato created a series of innovative tools that immediately impacted revenue for his firm. It didn’t take long to realize the potential of these applications beyond the scope of internal use, and in 2001, Broadlook Technologies was born.

"Linkedin is not a social network. Linkedin is a social database."

Stay tuned… a complete transcript of Donato’s podcast coming soon!

I was inspired to write this blog after reading Glen Cathey’s (The Boolean Black Belt) Article on Why Do So Many ATS Vendors Offer Poor Search Capability.  The article made me think about search engines (google, yahoo, etc) and how semantic search is being used with them.

What is semantic search?  To put is simple: semantic search can take, as input, a word like “Java” and offers up other related terms like “J2EE” or “Beans” (both are related to Java).  This allows the user to type in a few terms but match many, many terms.

The matching terms are built into an “expert system” that is continually built over time.  Many fancy names are given to these systems, based on how they are built, but basically they are sets of rules.

Semantic search is not AI (artificial intelligence).  If you hear that, it probably started in a marketing department somewhere.

Companies that have built semantic search engines, while they have not created AI, have spent a tremendous amount of time and resources to build these sets of rules.  The better engines can build rules on the fly from a new set of data, like resumes.  This is very cool stuff.

Overall, I like semantic search.  It has great potential, however, it has great weaknesses if used incorrectly.   If built into the engine itself, semantic search can be very powerful,  this is because semantic processing is done at the search engine side, without any limitations or constraints.  However, if bolted onto a search engine, it can be more harmful than good.

Here is what I mean.  I’ll try to keep my logic simple.

1. The Google search engine has a limit in how many terms can be submitted to it.

2. Semantic search, by it’s nature, creates permutations upon given terms. For example:

“Senior VP of Sales”  can be “SVP Sales” or “Senior Vice President of Sales”

to translate that into a boolean expression you get

“senior vp of sales” OR “SVP sales” OR “senior vice president of sales”

3.  After creating permutations upon several concepts, you are out of search terms.

I’m a big believe in laws (maybe not speed-limit laws), but more the “laws of the universe” type stuff.  I like to understand and deconstruct the rules and see if each one stands alone, or, do I need to recheck my premises.  In this spirit, just before the first sourceCon conference, I developed the Seven Laws of Internet Research.  I felt there was too much emphasis on memorizing search strings and the latest search engines or sites, but not enough fundamental thought leadership on how to think about searching the Internet.

The first two laws are

1. The Law of Permutation
2. The Law of Completeness

The Law of Permutation simply states that when searching the Internet, as it is not a homogeneous source of data, you must describe what you are looking for in the language of the many vs. the language of the one.  (YES, this is what Semantic search is doing).

The Law of Completeness states you must strive for completeness of search engine results in order to have the superior outcome

Big Question:  What happens if semantic search is applied before you reach completeness of results?

Answer:  Missing data. Competitors eat your lunch.  If you are a sales person, it means missed sales leads, if you are a recruiter, it means missed resumes or passive candidates.

Does this mean that I am anti-semantic search?  No way.  I think it has great potential.

Here are my take-aways:

-Semantic search should be inside the search engine for optimal results

-Semantic search bolted onto a standard search engine is severely limited.

-Semantic search will cause data to be missed if applied before reaching completeness of possible results

-When combining a standard search engine and semantic search, it is best to apply the semantic processing AFTER completeness of data has been reached.  In reality, this would not be semantic search, but semantic filtering.

Having worked with many databases as well as having extensive experience in searching the Internet, I thought I’d share some thoughts on the differences between the two.

When I observe people searching the Internet, there is a common mistake I see them making.  Most people search the Internet like they are searching a database. Don’t get me wrong,  the Internet does include databases. Thomas Register, Spoke and Zoominfo are examples of different types of databases. Via different methods, information is added to these data sources and some sort of query mechanism is provided the subscribers. Can you use the Zoom query on Thomas Register and visa versa? No, these are proprietary systems that have search methods specialized to the content inside them. Each of these databases is limited, incomplete, but stored in a homogonous fashion.

The Internet, in its entirety, is not homogonous.

Once data is exported into a CRM or Applicant Tracking System, it becomes homogonous.

The Internet, in its entirety, is not homogonous.

I hope I am getting this point across. The reason to reiterate this concept is to educate as to the 2 modes that you may have to operate in. Don’t treat the Internet like a database, and, conversely, don’t treat a database like the Internet. Some concepts are similar and some are very different.

Tips for searching a database:

-Learn the rules. You may not be able to use certain Boolean operators (AND,NOT, OR). Each database has different methods and capabilities.
-Know the limitations. You may only be able to export 10,000 records per year. You may only get back the first 100 matching records even though you thousands that match your search.
-Have a general idea as to the size and content within the database. This will allow you to properly set expectations for structuring your queries.
-Understand the normalization of that database and use it to your advantage. For example if all titles in the database are normalized to the short form (i.e. VP vs. Vice President), then your queries should all be in the short form.

Tips for searching the Internet

-Learn the rules. Just like searching a database, each search engine has different capabilities. (AltaVista allows a much longer search string than Google does).
-Know the limitations. (i.e. Google returns 1000 max results ).
-Don’t try to label what you are searching for with your terminology. Learn how the Internet is describing what you are looking for and then structure your search query.
-Understand that you will reach a point of diminishing returns. The Internet does not have an end. If you keep searching, you will probably continue to find more information.
-Understand that the Internet is not normalized.

Here is the inside scoop:  Technology people have a secret club, complete with handshake and everything.  It’s a club that we don’t want outsiders in.  So we create these long phrases that make peoples eyes glass over.  Why?  Because if everyone understood what we do, then we wouldn’t make the big bucks.   Being a recovering technologist, I’m on a continually journey to lose my geek speak. So get ready, here is the skinny on Data Normalization

First, understand that any organization that uses a CRM,  Applicant Tracking System or database of any sort will lose significant productivity if you don’t have a data normalization plan.   It is important.

Now for data normalization in a nutshell.  Look at the following list of company names.

Container Company
Container Co
Container Co.
The Container Company
The Container Co
The Container Co.
Container Company, The
Container Co, The
Container Co., The
Container Company Incorporated
Container Company Inc.
The Container Company Incorporated
The Container Company Inc
The Container Company Inc.
Container Company Incorporated, The
Container Company Inc, The
Container Company Inc., The
Container Co Inc, The
Container Co. Inc, The
Container Co. Inc., The
Container Co Inc., The

Did you notice that this is the same company?  If you don’t have a “DATA PLAN” this is what the inside of your database looks like.  Why?

I’m going to pick on recruiting:  Impatient, type-A personalities that all want to do things their own way.  Put 10 in an office and they are all going to follow there own path and your database will look like the above.

Here are some questions to ask yourself:

• Does your database include a large percentage of duplicates (over 20%)?
• When you search for a company, are their multiple entries, with notes scattered across many duplicates?
• Does your CRM allow you to enter in company names in any format you choose?
• Have you ever had 2 people in your company working with the same company and NOT knowing because they were saving notes under different company records?

If you answered YES to any of the above, you have “data nightmare”

How you can avoid a data nightmare

1. Create a “data plan”.   A  Data Plan consists of a set of executive decisions which controls how data will be treated.  This can be as simple as you will always use “Inc” instead of spelling out “Incorporated” when entering company names.
2. Educate your team about your Data Plan and data entry guidelines
3. Post your data entry guidelines
4. Enforce your data entry guidelines with internal policies.  How?  You don’t follow policy, you don’t get your commissions.  Try it, it works!
5. Enforce your data entry guidelines with technology.  The BEST way to do this is if your CRM vendor allows you to automatically enforce how data is entered
6. Force external programs that communicate with your CRM to follow YOUR Data Plan.  For example, Broadlook Technologies Profiler product has the ability to choose a data normalization schema.

If you follow my tips, you can avoid having a data nightmare.

In my years in the industry, I’ve worked with many vendors on the consulting side to help reduce the possibility of CRM failure.  While there is a whole host of reason that failure occurs, I have a very unique perspective into one of those reasons.  The Nature of Contact Information.

The nature of contact information is fairly finite (i.e. Company, URL, Name, Title, Email, Phone, Social Network membership, etc). In addition, the concept of contact information is a simple one to grasp. It is so simple, in fact, that if often gets overlooked.

One of the most important concepts in business is “be brilliant at the basics”. If you are brilliant at your basics many more complex processes will fall naturally into place. So how are you treating contact information?

The miss-handling of contact information can lead to dire consequences across your company.

Take the following work flow as an example:

Company X has the following attributes:

-10 sales reps
-Each sales rep enters 10 new contacts per day into their CRM system.
-Hand entering contact information takes an average of about 2 minutes per contact.

What does this equate to?

-20 minutes per day, per person, manually entering contact information.
-100 minutes per week, per person, manually entering contact information
-At 48 selling weeks per year, 2 weeks per year, per person, entering contact information.
-Organizationally that equates to 20 weeks per year entering contact information
-At 48 selling weeks per year, that means that 1/24th of each persons time or approximately 4% selling time is taken up doing data entry.   How many sales teams would like to boost performance with an extra 4% of selling time

The above is a dumb-simple illustration of metrics.  However, there are some additional complexities to take into account.

-Humans are fallible.  20% of records hand entered into a CRM or ATS are miss-keyed.
-Humans take shortcuts.  Most people enter the absolute basics: name, email, phone.  Usually full titles, locations, full company names, etc  are skipped.
-Miss-keyed information causes addition errors and degradation in data quality.
-People get tired.  Towards the end of the day, the 2 minutes that it takes to enter a record becomes 3 or 4.
-People don’t usually look up the record in their CRM first before entering it.  Gotta love duplicates!
-We all have our own style.  You say  “The Container Company”,  I say “Container Company, The”.  Again more duplicates!    The scary thing is that there are over 20 ways to write “The Container Company”.  I’ll be writing about that in my next blog about “Data Normalization”
-We all get interrupted.  This causes  a break in the work flow, more keying errors, more duplicates, less homogony.

Fast forward: Does this look like your company?

-You have massive errors in your CRM.
-Over 30% of your records are duplicates.
-Sometimes sales reps are working on the same accounts and they didn’t know it.
-Reporting is a nightmare, you want to do it, you really want to, but, because of the duplicates, reporting is inaccurate
-Once every few years or so, you make it “your mission” to remove the duplicates from your system.
-You, and your team, keep manually typing in contact information and
-You don’t have plan for data entry.  Every person wings it with their own unique style,  adding as many duplicates are their are unique personalities.

It would really suck if I stopped right here and basically left it that everyone is simply screwed as it relates to their CRM.

I won’t do that to you.  I have a solution.

Most of the above problems can be avoided by doing 2 things.

1.  Decide on how you want your organization to treat contact information (called a “Data Plan”).
2.  Remove the entering of contact information from humans and automate it.

#1 is simply accomplished by making a decision.
#2 can be done by getting a copy of Contact Capture (it is a free program)

What is Contact Capture?  It is a free program that automates that capture of contact information.  It can take that contact information, normalize it,  check for duplicates and then export it into your CRM.  No human data entry errors, no duplicates, no different forms of the same company name.  Problem solved.

Did I mention that Contact Capture is a FREE program?

Yes it captures single contacts, it captures lists of contacts.  It is by far the best technology on the planet for day-day contact information entry.

Many of you know about Contact Capture.  Historically,  Broadlook charged for it.  About a year ago we made it free and not enough people know about it.  So spread the word,  it is free, free, free.  Why?  Because we can and it is simply good business.   Contact Capture works with over 100 CRM systems.   While I don’t usually plug products in my blog, I feel that this is a no-brainer.   If anyone… ever… types in my contact information again, and gets my name wrong,  don’t blame my parents!

Get your copy of Contact Capture here

If Contact Capture is not compatible with your CRM, contact your CRM vendor and ask them why?

While writing this blog, I realized that there was a lot more than I wanted to cover here, so I wrote a 23 page white paper that started with this blog.  Get the Contact Capture white paper here.

Here is the link to put in your browser:

http://www.broadlook.com/pdfs/a-case-for-implementing-contact-capture-broadlook-technologies-white-paper.pdf