Real Estate API vs Web Scraping

A scraper is coupled to the exact HTML of the page it targets. It reads a div here, a class name there, a JSON blob embedded in a script tag. None of that is a contract — it is an implementation detail the site can change without warning, and property portals change it often.

Three failure modes dominate:

• Markup drift. A redesign, an A/B test, or a framework migration renames the selectors your parser depends on. The scraper does not error loudly — it silently returns empty or partial records, and you find out when someone downstream notices the data looks wrong.
• Client-side rendering. Modern portals render listing content in the browser via JavaScript. A requests + BeautifulSoup scraper sees an empty shell, so you graduate to headless Chrome (Playwright/Puppeteer), which is heavier, slower, and easier to fingerprint.
• Lazy-loaded and gated fields. Phone numbers behind a "reveal" button, descriptions loaded on scroll, details available only after login. Each one is a separate special case in your extraction code.

Teams that run this in production report spending a large share of their scraping engineers' time on defensive maintenance — repairing selectors and chasing regressions — rather than shipping product. That work produces no feature and no differentiation. It just keeps yesterday's data flowing.

Property portals have invested heavily in bot mitigation, because scraped listings directly undercut their business. What you are up against in 2026:

• Behavioral bot detection (e.g. DataDome, Cloudflare Bot Management) fingerprints the browser, inspects TLS and HTTP/2 signatures, and scores mouse movement, scroll speed, and request cadence. A naive headless browser is flagged in minutes.
• CAPTCHAs and JS challenges interrupt automated sessions. Solving services add per-thousand cost and latency, and they do not solve the underlying detection.
• IP reputation. Datacenter IPs are blocked on sight, so you move to residential proxies billed per gigabyte — a recurring, volume-scaling cost that only grows as you add coverage.
• Silent throttling and shadow-bans serve you degraded or stale data instead of an obvious 403, so you cannot even trust a "successful" response.

This is an arms race, and the sides are not evenly matched. The portals have larger security budgets than your internal scraping team, and every countermeasure you build is one release away from being obsolete. You are renting fragility.

A data API inverts this: you make an authenticated request to one stable endpoint and get structured data back. There is no fingerprint to hide, no proxy pool to rotate, no challenge to solve.

curl -H "X-API-KEY: <api_key>" \
  "https://api.stream.estate/documents/properties?includedDepartments[]=/departments/75&propertyTypes[]=1&transactionType=0&itemsPerPage=30"

Scraping is not automatically illegal, but scraping commercial real estate portals sits in a genuinely risky zone in the EU, and "the data was publicly visible" is not the defense engineers assume it is.

Three layers stack up:

• Terms of service. Every major portal's ToS explicitly prohibits automated extraction. Breaching them is a contractual and, depending on how access is obtained, potentially a liability question.
• The EU sui generis database right. Under Directive 96/9/EC, a party that makes a substantial investment in compiling a database can prohibit extraction of substantial parts of it. Listing portals are close to the textbook example of a protected database, and courts in several EU jurisdictions have ruled on systematic extraction of classified-ad databases.
• GDPR. Real estate listings routinely contain personal data — advertiser names, phone numbers, sometimes addresses. Re-using scraped personal data for your own commercial purposes triggers GDPR obligations regardless of the data being "public." EU regulators have been explicit that public visibility does not equal free re-use.

For a startup raising a round or a company going through due diligence, an undocumented scraping pipeline is a liability that a careful acquirer or investor will flag. Using a compliant data provider moves the sourcing responsibility to the vendor: Stream.Estate is EU-hosted and GDPR compliant, and you consume data through a licensed API rather than extracting it yourself.

This is not legal advice — talk to your counsel about your specific case. The point is that the legal question is real, not theoretical.

The reason scraping looks cheap is that most of its cost is never booked against it. A realistic total cost of ownership includes:

• Build. A production multi-portal scraper for a market like France is typically 3–6 months of senior engineering before it is reliable — anti-bot handling, rendering, retries, deduplication, monitoring.
• Maintenance. It never stops. Sites change, and someone has to keep the pipeline alive indefinitely. This is the largest and most under-counted cost.
• Infrastructure. Residential proxies (billed per GB), headless-browser compute, CAPTCHA-solving services, storage.
• Data cleanup. Scraped data is not analysis-ready. Your data team spends real time deduplicating the same listing across portals, normalizing field names (a "T3" here is a "3 rooms" there), and reconciling surfaces and property types.
• Opportunity cost. The engineers babysitting scrapers are the same engineers who could be building your actual product. This is the most expensive line and it never appears in a budget.
• Downtime risk. When a scraper breaks on a Friday night, your product shows stale listings and loses credibility until someone fixes it.

Add it up and an in-house scraping operation for broad national coverage runs into the hundreds of thousands of euros per year — and that spend does not buy reliability. See the cost breakdown table below.

The value of an API is not just "you don't scrape." It is that a whole category of work disappears from your codebase.

One endpoint, many sources. Stream.Estate aggregates 1,500+ sources — portals and agency sites — behind a single REST API. Adding a source is our problem, not a new scraper for you. Coverage today is France (primary) and Belgium, expanding progressively across Europe.

Deduplicated and normalized. The same listing published on several portals is reconciled into one record with consistent fields. 50M+ deduplicated listings, structured as Hydra JSON-LD, so you skip the cleanup layer entirely.

Real-time change tracking via webhooks. Instead of diffing daily snapshots yourself, you create a saved search and subscribe to events. Price changes, new ads, expirations — pushed to you.

import requests

# Search: houses for sale in Paris (dept 75), newest first
resp = requests.get(
    "https://api.stream.estate/documents/properties",
    headers={"X-API-KEY": "<api_key>"},
    params={
        "includedDepartments[]": "/departments/75",
        "propertyTypes[]": 1,          # 1 = House
        "transactionType": 0,          # 0 = Sell
        "orderByCreatedAt": "desc",
        "itemsPerPage": 30,            # max 30 per page
    },
)

for p in resp.json()["hydra:member"]:
    print(f"{p['title']} — {p['price']}€ — {p['surface']}m² — {p['city']['name']}")

# Get notified when prices change, instead of re-scraping
requests.post(
    "https://api.stream.estate/searches",
    headers={"X-API-KEY": "<api_key>"},
    json={
        "includedDepartments": ["/departments/75"],
        "propertyTypes": [1],
        "transactionType": 0,
        "eventEndpoint": "https://your-app.example/hooks/stream-estate",
        "subscribedEvents": ["property.ad.create", "ad.update.price"],
        "notificationEnabled": True,
    },
)

99.9% uptime SLA, < 200ms average latency. No proxy pool, no CAPTCHA budget, no Friday-night pages. The full endpoint surface is documented at docs.stream.estate.

You do not have to rip out scrapers on day one. Run the API alongside them and cut over gradually.

1. Audit what you actually consume. List the sources you scrape, the fields your product truly uses, your daily volume, and the transformations you apply (dedup, normalization, geocoding). Teams routinely find that a large share of their scraping code exists only to fix problems a structured API removes natively.
2. Proof of concept (1–2 weeks). Point the API at a narrow scope — one city, one property type — and compare coverage, freshness, and quality against your scraper output side by side. Start free, no credit card required.
3. Cut over source by source. Adapt your pipeline to consume the API response, then migrate the most unstable scrapers first — the ones that break most often and cost the most to maintain.
4. Decommission. Once coverage is confirmed, turn off the remaining scrapers, drop the proxy and CAPTCHA contracts, and reassign the engineers to product work.

A typical migration lands in two to three months with no service interruption. See pricing or start from the country coverage page for France.

Scraping is a legitimate tool, and pretending otherwise would be dishonest. It is often the right call when:

• You need a one-off dataset, not a live feed. A single market study, a research snapshot, or a due-diligence pull that you run once and never maintain. There is no ongoing breakage cost if there is no ongoing scraper.
• You are prototyping. Validating an idea over a weekend, where a fragile script that works today is fine because you will throw it away before it matters.
• The source is niche and unaggregated. A small regional site or a specialized listing source that no API covers. If the data only exists there, scraping it may be the only path.
• Volume is tiny and the target is not defended. A handful of pages from a site with no bot mitigation, run occasionally and politely (respecting robots.txt and rate limits).

The trade-off flips once the work becomes continuous, at scale, against defended sites — which is exactly the profile of production real estate data. At that point you are no longer writing a script; you are running an operations team against a hostile, moving target. That is where an API stops being the expensive option and becomes the cheap one.

Even then, verify the legal footing for your use case with counsel — the technical convenience of scraping does not change the ToS, database-right, or GDPR questions above.