Three sources for one NFT listing, and why scraping is the worst
A proof of concept for scraping an NFT marketplace, which mostly became an argument for not doing it.
The README opens by telling you to check the terms of service and credits Zayne’s tutorial as the basis. Both accurate, and both point at the more interesting thing: scraping is the worst of three available options.
An NFT listing is derived from two places. Ownership, transfers and on-chain sale events are chain state, readable by anyone from logs with no permission and no rate limit beyond your provider’s. The marketplace’s own index holds the rest, current asking prices, off-chain orders, collection metadata, curation.
So there are three sources for what looks like one dataset. Reading events from the chain is the most durable and gives the least. The marketplace’s JSON API, where one exists, gives most of it with a versioning contract. Scraping the rendered DOM gives everything and breaks whenever a class name changes, which for a frontend under active development is weekly. The only data genuinely requiring the marketplace is the off-chain order book.
Crediting the tutorial it came from is the part worth continuing to do.