Page 1 of 2 12 LastLast
Results 1 to 10 of 18

Thread: Concept for MLS import

  1. #1
    Join Date
    Dec 2003
    Posts
    410

    Default

    This is a concept for a MLS import module that could be adapted to your particular circumstances. The concept is based on several assumptions. I encourage feedback about these assumptions.

    1. Most, if not all, state IDX/MLS services make the MLS listings available via ftp download.

    2. The data files in the download will be in a database or other type format that can be manipulated and exported to a database.

    3. The IDX/MLS will use a logical naming protocol for the images associated with its property. For example: in my local IDX the image associated with the property data is simply the mls number (#######.jpg) plus '.jpg' and is in the MLS local area directory.

    note: my local IDX covers Florida, Maine and I believe So Dakota.



    Developing the Module:

    Part 1: (This part will vary depending on the MLS property data file)

    a) Map the fields in the MLS property database to the OR listing tables

    b) Using the naming protocol for the images construct the url that contains the associated image of the property.

    c) Upload all the fields that will be used including the field for the image url to the corresponding OR fields.


    Part 2:

    Modify the appropriate OR php file(s) so that the image contained in the image url (from 'b)' above) will create and display the thumb associated with it and for the thumb to link to the main image url.

    Part 3:

    Make as much of the above as automatic as possible.

    Parts 2 and 3 are independent of the MLS/IDX. Part 1 will have to be modified to accommodate the format that the MLS data file comes in.

    I encourage your feedback. The better we are able to describe and organize the module development the easier it will be for a programmer to code.

    To Do: make sure that the above is sufficient to develop the module and for a programmer willing to take on the project.

    One thought if someone might be willing to support the project through individual small PayPal contributions to the programmer. Another is to put the project out to bids.

    Frank

  2. #2
    Join Date
    Dec 2002
    Posts
    1,883

    Default

    Quote Originally Posted by frobn
    This is a concept for a MLS import module that could be adapted to your particular circumstances. The concept is based on several assumptions. I encourage feedback about these assumptions.

    1. Most, if not all, state IDX/MLS services make the MLS listings available via ftp download.

    2. The data files in the download will be in a database or other type format that can be manipulated and exported to a database.

    3. The IDX/MLS will use a logical naming protocol for the images associated with its property. For example: in my local IDX the image associated with the property data is simply the mls number (#######.jpg) plus '.jpg' and is in the MLS local area directory.

    note: my local IDX covers Florida, Maine and I believe So Dakota.



    Developing the Module:

    Part 1: (This part will vary depending on the MLS property data file)

    a) Map the fields in the MLS property database to the OR listing tables

    b) Using the naming protocol for the images construct the url that contains the associated image of the property.

    c) Upload all the fields that will be used including the field for the image url to the corresponding OR fields.


    Part 2:

    Modify the appropriate OR php file(s) so that the image contained in the image url (from 'b)' above) will create and display the thumb associated with it and for the thumb to link to the main image url.

    Part 3:

    Make as much of the above as automatic as possible.

    Parts 2 and 3 are independent of the MLS/IDX. Part 1 will have to be modified to accommodate the format that the MLS data file comes in.

    I encourage your feedback. The better we are able to describe and organize the module development the easier it will be for a programmer to code.

    To Do: make sure that the above is sufficient to develop the module and for a programmer willing to take on the project.

    One thought if someone might be willing to support the project through individual small PayPal contributions to the programmer. Another is to put the project out to bids.

    Frank
    I think wee should all make this MLS thing happen. In the long run all of us will benifit. I think we should take O-R to the hieghest level we can.

    Sugestion
    Set up a poll to see how much people is willing to make this happen. From the poll of those who are willing contibute to this we should pay the person(s) to do it with that contribution.

    Set up a paypal account just for this (I suggest Ryan do this part).
    Lets Get MLS into O-R once and for all.

    This does'nt have to take long. lets's unleash the power of O-R.

    Take the poll. When we get the number of who wants it, we split the cost and get it done. Let's make a mod for MLS import. It will do good for all of us.


    Now everyone should try and contibute, we all need this.

  3. #3
    Join Date
    Jul 2003
    Location
    California
    Age
    42
    Posts
    4,229

    Default

    Um.. guys... the_sandking has already created an MLS import Module that he charges a fee for.

    I haven't needed an MLS import yet... as my wife's broker refuses to share listings and thus his agents can't put other companies listings on their websites etc...

    However, when we do finally get through to him to share his listings I'll be going to sandking for his import module. I know he does good work and I'm sure whatever he charges for the module is reasonable considering the complexity of it and the need for it to be slightly changed for each mls out there.

    If you do plan on developing your own free module for everybody I'm all for that but, I think this is a mod that will have a slightly too high of learning curve for the average typical user to get properly setup on their own, thus resulting in MANY requests for help from the devs to help them set it up for their MLS.
    Open-Realty 2 Documentation

    ~The Difference Between an ORDEAL and an ADVENTURE is ATTITUDE~

  4. #4
    Join Date
    Dec 2003
    Posts
    410

    Default

    Mick wrote:

    Um.. guys... the_sandking has already created an MLS import Module that he charges a fee for.
    Mick I agree that for many people the best way to go will be to purchase a proprietary customization. Form what I understand Sandking has put in a lot of time and developed a excellent module. The only problem I have with a proprietary customization is that when the next upgrade comes it may break. From my perspective, an OR community MLS standard module will be easier to maintain and less expensive to maintain.

    A quest wrote:

    Take the poll. When we get the number of who wants it, we split the cost and get it done. Let's make a mod for MLS import. It will do good for all of us.
    I appreciate your thoughts. I think the process can be straight forward. If a programmer wanted to take on the project he/she could state his intentions and set up his own PayPal account. Those of us who are willing to contribute a small amount, say $5 - $15, or more, could do so. It would be a gamble for the programmer but one that I think would pay off. I have also had experience putting projects out for bids and have found that someone spots a quick, and usually elegant, way to do something and makes a reasonable bid. I have not been disappointed. If I can get a reasonable bid I will pay cost myself.

    I agree "Let's make it happen."

    Frank

  5. #5
    the_sandking's Avatar
    the_sandking is offline hadron remnant - Moderation Fachmann
    Join Date
    Apr 2003
    Location
    Nullspace
    Posts
    5,289

    Default

    Quote Originally Posted by Mick

    If you do plan on developing your own free module for everybody I'm all for that but, I think this is a mod that will have a slightly too high of learning curve for the average typical user to get properly setup on their own, thus resulting in MANY requests for help from the devs to help them set it up for their MLS.
    Thanks for the praise Mick!

    Somehow you always seem to hit the nail on the head when it comes to describing potential problems that *do* occur. In a nutshell, supporting folks costs $$ even when the app is "free".
    "Much of what looks like rudeness in hacker circles is not intended to give offense. Rather, it's the product of the direct, cut-through-the-BS communications style that is natural to people who are more concerned about solving problems than making others feel warm and fuzzy."

    "We gotta' go to the crappy town where I'm a hero!"
    -Hoban 'Wash' Washburne 2485-2519


    "When you’re born you get a ticket to the freak show. When you’re born in America, you get a front-row seat.."
    -George Carlin 1937-2008

    New to Open-Realty® and need help? Check the:
    -OR DOCUMENTATION -

    Important: Read this at least once in your lifetime
    How To Ask Questions The Smart Way

  6. #6
    the_sandking's Avatar
    the_sandking is offline hadron remnant - Moderation Fachmann
    Join Date
    Apr 2003
    Location
    Nullspace
    Posts
    5,289

    Default

    Since you are still in the early organizational stages of this project. I would like to toss in my $.02 given what I have already experienced after spending 100's of hours working with IDX.

    Forget IDX and do a RETS client instead. Then go back and do IDX (if you still want to)

    Some reasoning:
    -----------------------
    1) RETS is what the big-boys use, and because of that, it utilizes STANDARDS. (search on RETS in the OR forums for links)

    2) Development materials are freely available.

    3) Many professional developers, actively supporting the ongoing development via mail-list.

    4) RETS is a Client/Server app (see #1 again)

    5) the_sandking's conspiracy theory regarding IDX and exactly why it's not standardized. http://www.open-realty.org/phpBB2/vi...c&start=15

    So, there you have it.. At least 4 good reasons to do RETS instead...

    The project's anticipated timetable should also be carefully considered before throwing any $$ at it.. In other words, how long will an IDX MOD still be useful after the time it takes to develop, and before something else replaces IDX.

    However, it is also very understandable why IDX is important to many of you right now, considering so many "technologically backward" MLS's have only recently placed IDX data online, creating a short-term "market"..

    RETS might not be the long-term answer here either. Gambling is still gambling after all..

    O.K. It turned out to be $.03 but who's counting..

    Beerz!
    "Much of what looks like rudeness in hacker circles is not intended to give offense. Rather, it's the product of the direct, cut-through-the-BS communications style that is natural to people who are more concerned about solving problems than making others feel warm and fuzzy."

    "We gotta' go to the crappy town where I'm a hero!"
    -Hoban 'Wash' Washburne 2485-2519


    "When you’re born you get a ticket to the freak show. When you’re born in America, you get a front-row seat.."
    -George Carlin 1937-2008

    New to Open-Realty® and need help? Check the:
    -OR DOCUMENTATION -

    Important: Read this at least once in your lifetime
    How To Ask Questions The Smart Way

  7. #7
    Join Date
    Dec 2003
    Posts
    410

    Default

    sandking wrote:

    Since you are still in the early organizational stages of this project. I would like to toss in my $.02 given what I have already experienced after spending 100's of hours working with IDX.

    Forget IDX and do a RETS client instead. Then go back and do IDX (if you still want to)

    Some reasoning:
    -----------------------
    1) RETS is what the big-boys use, and because of that, it utilizes STANDARDS. (search on RETS in the OR forums for links)

    2) Development materials are freely available.

    3) Many professional developers, actively supporting the ongoing development via mail-list.

    4) RETS is a Client/Server app (see #1 again)

    In another thread greengiant wrote:

    But my MLS uses IDX or RETS Standards so why isnt there a free script to import that data? RETS and IDX are not standards. YEs you heard me they are not standards. Standards imply that you can transfer data to someone and they can follow a documented path and retrieve the data. Guess what, you can't. IDX and RETS are "standards" in the fact they saw data must be able to be transfered and we are goign to use xml to do it. THey do not saw these are teh fields you have to have and this is what we will call them. That is all left up to the local MLS. So there is no way to build a one size fits all solution.
    This is exactly the type of dialogue I was hoping for. I am not a programmer but have a good knowledge of programming concepts. Sandking I follow your reasoning to build on standards, but are there standards? What I am trying to do is to take what is common to any data export/import and to code that part. For example, assuming that I can get the data into the OR tables it has to be usable whether it comes from xyz IDX or xyz RETS. If I download the images file from my IDX unzip it and upload them to the photo directory then I would not have to make any changes to the php files at all. But part of the project is to use the existing photos from the IDX which may (I am not sure) necessitate changes in the php files.

    The part of the project that is peculiar to each different IDX/RETS is to take the data in the form available and manipulate it so it can be imported into the proper OR tables and fields. Translating from one format to another may be tedious and time consuming but it is not difficult.

    Please, anyone, correct me or help me find a better approach otherwise the project will bomb.

    I have already receive two reasonable bids from programmers who want the project and have given a time frame of 2-3 weeks. Before going further I would appreciate feedback and advice from the community.

    Frank

  8. #8
    Join Date
    Jul 2003
    Location
    California
    Age
    42
    Posts
    4,229

    Default

    But part of the project is to use the existing photos from the IDX which may (I am not sure) necessitate changes in the php files.
    A problem with this would be that some MLS's have rules against doing this.

    Again going back to creating a solution that will work for everybody would require either coding for both options: 1 being able to use images off the MLS's system linked directly.... and 2: moving the images to your OR system to be used from there.

    I would forsee option 2 to be the more universal way to go since finding the path to the image files on your MLS's server may be tedious and some servers might have checks in place to prevent people from linking to their images from offsite (bandwidth stealing). These can make option 1 not an option for many people.
    Open-Realty 2 Documentation

    ~The Difference Between an ORDEAL and an ADVENTURE is ATTITUDE~

  9. #9
    the_sandking's Avatar
    the_sandking is offline hadron remnant - Moderation Fachmann
    Join Date
    Apr 2003
    Location
    Nullspace
    Posts
    5,289

    Default

    Quote Originally Posted by frobn
    Sandking I follow your reasoning to build on standards, but are there standards?
    RETS is indeed a standard, v1.5 of the RETS specification is being developed right now. Look for those links here in the forums.. Lots of info there.

    IDX was only a MANDATE from the NAR (the King) to make the MLS's (Prince's ) data freely available to the AGENTS (the peasants) for thieir benefit. The king was not specific when he made this order, so the peasants get whatever their local Prince has cobbled together and made available.

    With RETS the SERVERS and CLIENTS may differ in various ways, but the data format i.e. field names, types and lengths, are consistant regardless of which MLS the data came from. RETS is mostly driven by the vendors that make the software that the MLS's use.

    However, given how unpredictible Kings can be, the King can always change his mind... Bypassing the Prince altogether perhaps...
    "Much of what looks like rudeness in hacker circles is not intended to give offense. Rather, it's the product of the direct, cut-through-the-BS communications style that is natural to people who are more concerned about solving problems than making others feel warm and fuzzy."

    "We gotta' go to the crappy town where I'm a hero!"
    -Hoban 'Wash' Washburne 2485-2519


    "When you’re born you get a ticket to the freak show. When you’re born in America, you get a front-row seat.."
    -George Carlin 1937-2008

    New to Open-Realty® and need help? Check the:
    -OR DOCUMENTATION -

    Important: Read this at least once in your lifetime
    How To Ask Questions The Smart Way

  10. #10
    the_sandking's Avatar
    the_sandking is offline hadron remnant - Moderation Fachmann
    Join Date
    Apr 2003
    Location
    Nullspace
    Posts
    5,289

    Default

    From the RETS mailing list:
    > So can anybody give me some info so that I
    > can
    > start working on the project or tell me where i can
    > find more info about fetching RETS data.

    There are a couple of reference RETS client implementations floating around on the Internet, one of which is at http://www.rets.org/resources/index.html. To my knowledge, there is no reference implementation written in PHP.

    I would recommend downloading whichever RETS spec you will be programming to (1.01 or 1.5), printing it out, and reading through it.

    > How can i fetch data from site "http://rets.hicentralmls.com/" using a
    > scripting language(PHP/ASP/CGI).

    To create a RETS client in PHP I used:
    - The CURL library
    - The XML parser functions

    If you do not understand how HTTP works, i.e., if you cannot fetch a page using telnet, you should learn that first. It will help you immensely during the development cycle, especially during the LOGIN transaction.

    With the LOGIN transaction, some servers use the format specified in the 1.01 spec, others use the Digest authentication specified in the HTTP spec. If you plan on reusing your RETS library, be sure that it supports both types of authentication.
    "Much of what looks like rudeness in hacker circles is not intended to give offense. Rather, it's the product of the direct, cut-through-the-BS communications style that is natural to people who are more concerned about solving problems than making others feel warm and fuzzy."

    "We gotta' go to the crappy town where I'm a hero!"
    -Hoban 'Wash' Washburne 2485-2519


    "When you’re born you get a ticket to the freak show. When you’re born in America, you get a front-row seat.."
    -George Carlin 1937-2008

    New to Open-Realty® and need help? Check the:
    -OR DOCUMENTATION -

    Important: Read this at least once in your lifetime
    How To Ask Questions The Smart Way

Similar Threads

  1. MLS Import Project
    By frobn in forum Feature Request (CLOSED)
    Replies: 9
    Last Post: 03-30-2005, 05:16 PM
  2. mls import
    By virtualgeorge in forum Work
    Replies: 3
    Last Post: 10-01-2004, 03:46 AM
  3. XML Import Possiblities
    By lasor in forum HELP (CLOSED)
    Replies: 11
    Last Post: 07-30-2004, 07:17 PM
  4. IDX import util
    By Anonymous in forum Feature Request (CLOSED)
    Replies: 2
    Last Post: 01-18-2004, 02:36 PM
  5. import data
    By Anonymous in forum HELP (CLOSED)
    Replies: 0
    Last Post: 05-11-2003, 07:29 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •