NAR Calls Google A "Scraper Site" and Stops Allowing MLS To Be Crawled

By
Real Estate Services with Real Estate SEO Pros

May 12, 2009 at 2:09pm ET by Matt McGee

With support from the National Association of REALTORS® (NAR), the Indianapolis Metropolitan Board of REALTORS® (MIBOR) has forced some of its members to stop allowing certain MLS listings to be crawled and indexed by Google because Google (and other search engines) is considered a “scraper” site.

This is the latest episode in a long-running battle over who controls home listings that are part of the Multiple Listing Service, or MLS. Some of the affected real estate professionals plan to ask the NAR to change its opinion on search engines at its national convention this week.

Paula Henry is one of about 15 real estate agents affected by the NAR/MIBOR decision. Like many agents/brokers, she shows not only her own listings on her web site, but also listings from other agents and brokers who participate in a data sharing agreement through the Internet Data Exchange (IDX) system. There are strict rules over how listings from other agents may be displayed. In a post on AgentGenius.com, Paula explains the one that has led to Google’s classification as a scraper site:

Section 15.2.2 – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent ’scraping’ or other unauthorized accessing, reproduction, or use of the BLC database”

(BLC is the Indianapolis version of MLS.)

After apparently getting a complaint from another agent, MIBOR consulted with the NAR and the NAR confirmed that the rule above applies to search engines. On March 27, MIBOR sent Red Door Real Estate (where Paula Henry does business) a cease-and-desist letter detailing two issues:

  1. “…the National Association of REALTORS® (NAR) is in agreement with our interpretation of the policy that the above described practice of ‘indexing your Web site’ as you have called it, is a method of scraping or reproducing the data”
  2. “Under IDX policy … participants have no authority to advertise those listings [from other participants] in any other way, including Internet search engines”

The C&D letter asks Red Door to either use robots.txt (actually referred to as “rebot.txt” in the letter) to block search engines from crawling the listings on their sites that belong to other agents/brokers, or to remove the non-Red Door listings altogether. In her blog post, Paula writes that additional demands were made in April requiring Red Door to remove the MLS number, street address, and other listing data from the Page Titles and Meta Description tags of pages on their web sites.

Not mentioned in all of this is the fact that Google is practically running its own national MLS database, with the same kind of search and sort options (Price, Beds, Baths, Area, etc.) in Google Maps that you’d expect to find on full-fledged real estate sites.

goog-real-estate

Real estate brokers and agents submit their own listings to Google (or they have a third party do it), so the rules about how other agents’ listings are displayed don’t apply. And it’s not just about Google; Yahoo Real Estate also gets listings from agents and brokers, and places them on highly-optimized pages that, in my experience, often rank well for popular real estate search terms.

As for Google being a scraper site, Paula Henry will be one of two real estate agents speaking to the NAR on Thursday. She says they’ll ask the organization to review and change its policy so that Google (and other search engines) is allowed to index all real estate listings from the IDX system, no matter whose web site they appear on.


Matt McGee is the Search Engine Land Assignment Editor, and offers search marketing consulting and training to businesses of all sizes. He blogs at Small Business Search Marketing and HyperlocalBlogger.com.

Posted in Real Estate SEO NewsNo Comments

<!-- You can start editing here. --> <!-- If comments are open, but there are no comments. -->

close

Re-Bloggged 1 time:

Re-Blogged By Re-Blogged At
  1. Robert May 07/28/2009 09:09 PM
Spam prevention

Accessibility option: listen to a question and answer it!

To submit the form,
drag the tshirt to the circle on the side.

Type below the answer to what you hear. Numbers or words, lowercase:

Topic:
Real Estate Technology & Tools
Groups:
Getting Visible on Google. How To's
Search Engines, SEM, SEO & Top Search Engine Ranking.
Social Media Marketing
Technology Side of Active Rain
The Lounge at Active Rain
Tags:
google mls
nar blocks mls access
idx listings google
mls listings in search engines

Comments 5 New Comment

Anonymous
Post a Comment
Spam prevention

Accessibility option: listen to a question and answer it!

To submit the form,
drag the pants to the circle on the side.

Type below the answer to what you hear. Numbers or words, lowercase:

Rainmaker
993,502
Evelyn Johnston
Super Agent, Probably the Best Agent in the Galaxy
Elkhart County Subdivisions, LLC

Maybe I am not understanding the full picture here, but I would think it is a good thing that NAR is limiting who can have access to our information.  It will be protection for our profession.

On the other hand, if 80% of new home buyers search the internet and we have limited access to them, will they then call a real estate agent?  Unless they want to look at each agents website individually they would have to wouldn't they?

July 29, 2009 07:40 AM
Rainer
5,190
Real Estate SEO Pros Inc
Real Estate SEO Pros

I think the main takeaway is that in order for you to get your listings in front of the eyes of people, you will need to keep up with how people are searching for homes. It soon will not be enough to rely on the MLS in order to get your homes viewed. The protection for your industry IS slipping and there are challenges from every direction. The safe bet is to be prepared and work to be in a good position as the landscape changes. It will be interesting to see how it plays out

July 29, 2009 05:04 PM
Anonymous #3
Anonymous
john

I THINK WE HAVE ALLOWED THE REALTOR ASSOCIATION AND THE MLS TO HAVE TO MUCH POWER OVER OUR INDUSTRY , WE ARE MEMBERS , I REALLY HATE HOW THEY TURNED INTO A DICTATING COMPANY TELLING US HOW TO RUN OUR BUSINESS , AND THAT NOW MLS FEEL THEY HAVE THE RIGHT TO FINE US IS UNBELIEVABLE !

October 12, 2009 07:30 AM
Anonymous #4
Anonymous
Jonathan

I dont' think MLS is useful anymore. Most of the homes on MLS do not sell. It was a good idea before the internet.

October 28, 2009 03:54 PM
Rainmaker
172,245
Randy Landis
Retirement Consultant
Jimmy Langley Realty

This is just another example of how certain members of the real estate community and NAR attempt to 'protect' what they believe is their right to exclusivity.  I personally have had it with all the control and agree with John. Give me back the authority to do what is in my clients best interest: expose my clients listing. I'm sure that if you asked all the thousands of sellers whose properties are 'exposed', they would not have a problem with scraping. 

The only thing wrong with this entire situation is that third party recipients of for-sale data are re-selling it to agents through lead generation. This is where the focus needs to be...stopping unlicensed individuals from profiting from the sale of real estate.  A fiduciary responsibility we have with our clients is to support their best interests, and if my listings were being 'scraped' and exposed, then how is that not in the best interest of my client? Protectionism policies are not good for this country nor is it good for our industry. Buyers will locate their dream home most often on their own....let them choose their agent the same way - through research.

December 13, 2009 04:05 PM
Anonymous
Post a Comment
Spam prevention

Accessibility option: listen to a question and answer it!

To submit the form,
drag the house to the circle on the side.

Type below the answer to what you hear. Numbers or words, lowercase:

Rainer
5,190

Real Estate SEO Pros Inc

Ask me a question
*
*
*
Spam prevention

Accessibility option: listen to a question and answer it!

To submit the form,
drag the lock to the circle on the side.

Type below the answer to what you hear. Numbers or words, lowercase: