How to Price Web Scraped Data
5 Factors that Drive Data Pricing
About Data Boutique
Data Boutique is a web-scraped data marketplace.
If you’re looking for web data, there is a high chance someone is already collecting it. Data Boutique makes it easier to buy web data from them.
Join our Platform to learn and interact about this project:
A new way of buying and selling data
Yesterday we enabled data sellers to autonomously publish their data on Data Boutique. Anyone collecting data via web scraping can start offering it on the platform.
It’s a big deal, as we are breaking down barriers to safe and transparent web data adoption.
I think of it as the consumerization of B2B data transactions: We move from one-to-one private negotiations to an add-to-cart B2C-like philosophy. It is a new way of accessing data. It makes life easier, faster, and more profitable for buyers and sellers.
A question has been rolling around among the initial group of sellers, though: Since we move from a custom-project world towards a productized approach, how should data pricing be set?
5 Factors that Drive Pricing
Price needs to balance the seller’s costs with the buyer’s value. Here are the 5 factors that should be considered when opting for a price point.
Extraction Costs
It may seem obvious, but since many web scraping activities are done internally, not all actors have a clear, numerical understanding of their unit costs. Nonetheless, knowing if a single run of the scrape costs 2 USD, 20 USD, or 200 USD in variable costs (proxy networks, web scraping aiding tools, and platforms) and non-variable costs (coding hours) is critical.
Unit costs act as a floor for prices. And they also provide an excellent make-or-buy indicator: If your cost is 200 USD per run and you find it on Data Boutique for 20 USD, it might be a good time to switch to BUY.
If you’re so efficient and run it at 2 USD, it might be an excellent indicator to keep the MAKE and start selling on Data Boutique for 10X or 50X as much.
Self-Cannibalization Risk
If you’re already selling products based on the same data, you must be careful in considering the potential loss in sales caused by the introduction of a new product at a lower price point. Existing data products act as a floor in the pricing strategy, determining a minimum viable price.
A significant exception occurs when the existing product has a differentiating value from pure raw data, like history, charts, insight extraction, AI algorithms, and visualizations. Don’t underestimate this.
Example:
When a company provides rich data analysis featuring a dashboard, long history, and a high level of insight extraction, the new offering of raw web data does not represent a threat. The client would need to go through a lot if they wanted to switch to purchasing raw data and rebuild the entire value chain.
Adoption Curve
Data Boutique has an asymmetrical supply: Sellers provide data daily while buyers pay as they go. This increases service quality but makes it more challenging for the sellers at the start. Buyers have an adoption curve, especially when data offering is new. As we have been experimenting in our first months, a typical buyer's journey moves from an initial frugal approach to regular, scheduled updates.
Price needs to keep this in mind to make it sustainable.
Visible Competitors
More sellers can offer the same data, and each can set their own price. These are the competitors you can see. They are visible either on Databoutique or on other platforms. Buyers will evaluate sellers’ credibility, reliability, and price. Not price alone.
Also, prices of similar datasets play a role in orienting buyers’ expectations, but to a lesser extent.
Invisible Competitors
A final thought on the most muscular price ceiling that all sellers have to be aware of: The cost for a buyer to make the web scraper in-house. This is the competitor you don’t see, the invisible alternative that most cases will win, if pricing is not thought carefully.
Sellers, in fact, are not competing against each other: They compete against a buyer who has widespread access to web scraping talent and technology, that is de facto a commodity.
Join the Project
That was it for this week!
Data Boutique is a community for sustainable, ethical, high-quality web data exchanges. You can browse the current catalog and add your request if a website is not listed. Saving datasets to your interest list will allow sellers to correctly size the demand for datasets and onboard the platform.
More on this project can be found on our Discord channels.
Thanks for reading and sharing this.