Read an MLS RETS feed with PHP

RETS is a data format for retrieving real estate listings from an MLS. It is updated in real-time and uses its own query language to retrieve just the listings you want from the server. This article will walk you through connecting to a RETS server and retrieving listings using PHRETS.

Step 1: Get Credentials

You need to get access to the MLS feed. Apply for access with the MLS provider in the relevant area. There is usually a fee involved as well as contracts to sign and rules to read. At the end of this process you’ll have an access URL, username and password.

Step 2: Test Ports

You need to be able to connect from your server to the RETS server. This may be on a non-standard port. An easy way to test whether your firewall might be blocking access is to try to connect from your server to the RETS server using Lynx. For this example let’s say our RETS server is accessed via port 6666.

In the terminal from your server (or via SSH or Telnet) try:

lynx http://rets-server-url.com:6666

1	lynx http://rets-server-url.com:6666

If lynx can’t connect at all the firewall is a likely culprit. If it returns an error about not being able to login you’re good. That means it got through the firewall.

These instructions will help you open up a port on a Linux server using iptables. You can skip to the next section if you successfully connected with Lynx.

Allow access from the terminal

iptables -A OUTPUT -p tcp --dport 6666 -j ACCEPT

1	iptables -A OUTPUT -p tcp --dport 6666 -j ACCEPT

iptables -A INPUT -p tcp --dport 6666 -j ACCEPT

1	iptables -A INPUT -p tcp --dport 6666 -j ACCEPT

service iptables restart

1	service iptables restart

Note that iptables get reset whenever you reboot the server. You’ll probably want to set them more permanently, but that is outside the scope of this article.

Allow access from WHM

Plugins -> Config Server Security & Firewall -> Firewall Configuration

Scroll down to the TCP ports allow/deny section and add your port.

WHM CSF Firewall Configuration

Step 3: Data Structure

Ok now that you are successfully connected you can take a look at the data structure. I very slightly modified the basic connection script that comes with PHRETS.

<?php

$rets_login_url = "http://rets-login-url.com:6666";
$rets_username = "your_username";
$rets_password = "your_password";

//////////////////////////////

require_once("phrets.php");

// start rets connection
$rets = new phRETS;

// Uncomment and change the following if you're connecting
// to a server that supports a version other than RETS 1.5

//$rets->AddHeader("RETS-Version", "RETS/1.7.2");

echo "+ Connecting to {$rets_login_url} as {$rets_username}<br>\n";
$connect = $rets->Connect($rets_login_url, $rets_username, $rets_password);

// check for errors
if ($connect) {
        echo "  + Connected<br>\n";
}
else {
        echo "  + Not connected:<br>\n";
        print_r($rets->Error());
        exit;
}

$types = $rets->GetMetadataTypes();

// check for errors
if (!$types) {
        print_r($rets->Error());
}
else {
        var_dump($types);
}

echo "+ Disconnecting<br>\n";
$rets->Disconnect();

<?php

$rets_login_url = "http://rets-login-url.com:6666";

$rets_username = "your_username";

$rets_password = "your_password";

//////////////////////////////

require_once("phrets.php");

// start rets connection

$rets = new phRETS;

// Uncomment and change the following if you're connecting

// to a server that supports a version other than RETS 1.5

//$rets->AddHeader("RETS-Version", "RETS/1.7.2");

echo "+ Connecting to {$rets_login_url} as {$rets_username} \n";

$connect = $rets->Connect($rets_login_url, $rets_username, $rets_password);

// check for errors

if ($connect) {

echo " + Connected \n";

}

else {

echo " + Not connected: \n";

print_r($rets->Error());

exit;

}

$types = $rets->GetMetadataTypes();

// check for errors

if (!$types) {

print_r($rets->Error());

}

else {

var_dump($types);

}

echo "+ Disconnecting \n";

$rets->Disconnect();

What I get back is a dump that looks similar to this (I didn’t include everything for the sake of brevity):

[2]=>
  array(2) {
    ["Resource"]=>
    string(8) "Property"
    ["Data"]=>
    array(7) {
      [0]=>
      array(8) {
        ["ClassName"]=>
        string(19) "ResidentialProperty"
        ["VisibleName"]=>
        string(20) "Residential Property"
        ["StandardName"]=>
        string(19) "ResidentialProperty"
        ["Description"]=>
        string(3) "RES"
        ["TableVersion"]=>
        string(11) "000000"
        ["TableDate"]=>
        string(29) "Thu, 24 Apr 2010 15:43:24 GMT"
        ["UpdateVersion"]=>
        string(11) "000000"
        ["UpdateDate"]=>
        string(29) "Thu, 24 Apr 2010 15:43:24 GMT"
      }
      [1]=>
      array(8) {
        ["ClassName"]=>
        string(15) "ApartmentIncome"
        ["VisibleName"]=>
        string(15) "ApartmentIncome"

[2]=>

array(2) {

["Resource"]=>

string(8) "Property"

["Data"]=>

array(7) {

[0]=>

array(8) {

["ClassName"]=>

string(19) "ResidentialProperty"

["VisibleName"]=>

string(20) "Residential Property"

["StandardName"]=>

string(19) "ResidentialProperty"

["Description"]=>

string(3) "RES"

["TableVersion"]=>

string(11) "000000"

["TableDate"]=>

string(29) "Thu, 24 Apr 2010 15:43:24 GMT"

["UpdateVersion"]=>

string(11) "000000"

["UpdateDate"]=>

string(29) "Thu, 24 Apr 2010 15:43:24 GMT"

}

[1]=>

array(8) {

["ClassName"]=>

string(15) "ApartmentIncome"

["VisibleName"]=>

string(15) "ApartmentIncome"

To access residential properties on this feed the resource name is Property and the class name is ResidentialProperty

Now if I want to get meta data from that table I can use:

 $table = $rets->GetMetadataTable("Property","ResidentialProperty");

1	$table = $rets->GetMetadataTable("Property","ResidentialProperty");

If I var_dump($table) I get an array of available fields and related info. . Here’s a small sample:

 [0]=>
  array(28) {
    ["SystemName"]=>
    string(11) "AgentStatus"
    ["StandardName"]=>
    string(11) "AgentStatus"
...
[1]=>
  array(28) {
    ["SystemName"]=>
    string(12) "OfficeStatus"
    ["StandardName"]=>
    string(12) "OfficeStatus"
    ["LongName"]=>
    string(13) "Office Status"

[0]=>

array(28) {

["SystemName"]=>

string(11) "AgentStatus"

["StandardName"]=>

string(11) "AgentStatus"

...

[1]=>

array(28) {

["SystemName"]=>

string(12) "OfficeStatus"

["StandardName"]=>

string(12) "OfficeStatus"

["LongName"]=>

string(13) "Office Status"

I can loop through these and spit out the SystemName to get a list like this (just a partial list to show an example of the fields you’ll have to work with):

AgentStatus
OfficeStatus
AssociationFee
AvailableForLeaseYN
AverageMonthlyUtilities
BasementArea
BasementSqFtFinished
BasementPresent
Bath1Size
BathsTotal
Bedroom1Flooring
Bedroom1Level
Bedroom1Size
Bedroom2Flooring
Bedroom2Level
Bedroom2Size
Bedrooms
DateChange
ListDate
DatePhoto
DatePriceChange
ModificationTimestamp

Step 4: Search Listings

Now that I know my resource, class and fields names I can perform a search. The following code would return all listings for 2013. I set the optional limit to 20 since we’re just testing right now. Here are more details on PHRETS SearchQuery and how to format a query.

//$rets->SearchQuery ( string $resource, string $class, string $query [, array $options] )

$search = $rets->SearchQuery("Property","ResidentialProperty","(ListDate=2013-01-01+)", array("Limit" => "20") );
while ($listing = $rets->FetchRow($search)) {
        var_dump($listing);
}
$rets->FreeResult($search);

//$rets->SearchQuery ( string $resource, string $class, string $query [, array $options] )

$search = $rets->SearchQuery("Property","ResidentialProperty","(ListDate=2013-01-01+)", array("Limit" => "20") );

while ($listing = $rets->FetchRow($search)) {

var_dump($listing);

}

$rets->FreeResult($search);

Step 5: Images

We’ll be using GetObject and GetMetadataObject to get images.

First GetMetadataObject so we know the type of object to get.

I used the PHRETS example code to get the object type.

$object_types = $rets->GetMetadataObjects("Property");
foreach ($object_types as $type) {
    echo "+ Object {$type['ObjectType']} described as " . $type['Description'] . "<br />";
}

$object_types = $rets->GetMetadataObjects("Property");

foreach ($object_types as $type) {

echo "+ Object {$type['ObjectType']} described as " . $type['Description'] . " ";

}

This returned 2 object types: Photo and HQPhoto. Again, using an almost exact copy of the PHRETS example I grab all the images for property ID 12345.

Some points to note:

I got the property ID from a listing in the last step (a result of SearchQuery)
The directory where you write the images must have permissions to be writeable by the webserver

$photos = $rets->GetObject("Property", "Photo", "12345");
foreach ($photos as $photo) {
        $listing = $photo['Content-ID'];
        $number = $photo['Object-ID'];

        if ($photo['Success'] == true) {
                file_put_contents("property_images/image-{$listing}-{$number}.jpg", $photo['Data']);
        }
        else {
                echo "({$listing}-{$number}): {$photo['ReplyCode']} = {$photo['ReplyText']}\n";
        }
}

$photos = $rets->GetObject("Property", "Photo", "12345");

foreach ($photos as $photo) {

$listing = $photo['Content-ID'];

$number = $photo['Object-ID'];

if ($photo['Success'] == true) {

file_put_contents("property_images/image-{$listing}-{$number}.jpg", $photo['Data']);

}

else {

echo "({$listing}-{$number}): {$photo['ReplyCode']} = {$photo['ReplyText']}\n";

}

Step 6: Practical Use

One of the nice things about RETS is the query language. This allows you to get only the results you want from the server. The RETS feed is also updated in real time as opposed to others which are updated nightly. Still, it seems more practical to me to populate a database with the info for my web app. I plan to use CodeIgniter to both search and populate the db. I’ll create a cron job for the population and run it 1-4 times per day depending on what the client wants. The data is hand-entered by realtors (as mentioned in Drew’s tutorial) so I’ll probably do some clean-up on the data as it gets imported. That will be covered next time.

Resources and References

Drew Powers – Getting Started with RETS for PHP
Troy Davvison – PHRETS
FlexMLS – DMQL Tutorial (RETS query language)