Writing a Python Script to Scrape Telfar

Wednesday, June 30, 2021

A few days ago, a friend was telling me about Telfar. I had never heard the name before and I didn't know it was a fashion designer brand. This writeup is not about Telfar though. It's about copping a designer Telfar bag before it sells out.

As with all designer items, when they are out, they sell very quickly. My friend has never been lucky with copping a bag and I figured this was a problem that could easily be solved with code.

I spoke with a friend about it and we came to a few conclusions

  1. This is a scraping job: If you've never heard the term scraping, it just means to crawl or get information from a web page. I needed to find a way to know when a particular Telfar bag was up for sale and no longer tagged "Coming soon". Since there is no API, scraping is the next logical route.

  2. Use Python, BeautifulSoup and PythonAnywhere: Python was already top of mind because it's ridiculous easy to write a script that scrapes data from a web page with BeautifulSoup. To be honest, I had never heard of PythonAnywhere before speaking to my friend and I love the tool. It helps you run python anywhere. How awesome is that? And the best part is that you can write your code on the cloud :mind-blown. We took the platform for a test run, created a flask web app and it worked flawlessly. I was hooked! Best part is they have a mechanism to run your python script hourly or daily. Perfect for my usecase.

A worthy mention is Poetry. It's a package manager (think npm/yarn) for python. It helps with managing virtual environments, installing, updating and removing packages.

  1. This script had to run on a consistent interval: Every 15 mins, we trigger the script to crawl the page and determine if it's still coming soon or it's available for sale.

  2. Determine the Call to Action (CTA) when the bag is available for sale: I didn't want to automatically purchase the bag because I think it involved too much of a risk that isn't necessary so we thought of different CTAs that might suffice. Email, text message, slack message etc. We decided to go with email because for this application, it's the cheapest and least difficult to implement.

After having these basic ideas out, I spent a few hours before work on Monday writing the code to solve this problem.

One thing I really loved was the fact that I wrote tests. I didn't really follow TDD which is writing tests before writing code but my testing did influence the way I refactored my code. It's a lot easier to test a piece of code that is pure i.e for the same input, it'll always return the output

Here's a flowchart of the program

Here's the end result

Powered By Swish