Magpie QA
About Dale
I'm a versatile technology leader who loves tackling complex technical challenges and turning them into elegant solutions. My passion lies in bringing teams together, driving innovation, and creating transformative technical strategies that make a real impact. Whether I'm optimising systems, mentoring teams, or crafting new solutions, I bring enthusiasm and strategic thinking to every project I touch.
Experience
Magpie Technology
Data capture and digital shelf insights in the electronics and telecoms industries
Head of Technology
2023 - 2024
- Crafted and presented a technology vision and strategy for Magpie to the SMT, focused on reducing marginal scrape costs and refactoring the codebase for machine learning readiness. The strategy organised teams around key scraping functions—product discovery, data capture, parsing, and classification—to boost scalability and operational efficiency.
- Championed the creation of a Head of Infrastructure role, collaborated with the hire to enhance infrastructure observability, and improved server, proxy, and Chromedriver reliability.
- Developed a modular scraping approach using the Laravel container and dependency injection to separate concerns and boost reusability.
- Built a high-scale scraping proof of concept for US broadband data, successfully scraping a notoriously challenging site and contributing to the company’s acquisition by a US VC-backed firm.
Head of Data Delivery
2021 - 2023
- Led “Project De-Snail,” driving the technical strategy for concurrent scraping with Redis queues.
- Optimised scrapes, achieving up to 20x faster durations, and ensured weekly compliance scrapes completed in under 1 hour.
- Developed a high-scale scraping solution to track compliance for a Big Five company, ensuring product detail pages on a major retailer’s site were free from competitor advertisements.
Senior Software Developer
2020 - 2021
- Developed the Magpie QA Platform, enabling data quality analysts to efficiently audit data across multiple Magpie products.
- Integrated advanced features for automated issue tracking, streamlining quality assurance workflows.
- Wrote web scrapes with PHP WebDriver, DomCrawler, proxy rotation, and captcha solving.
a dozen eggs
Branding, design and web agency
Developer
2016 - 2020
- Built numerous highly performant WordPress sites and Woocommerce shops for clients.
- Worked closely with designers on web UI/UX, leveraging strong design communication skills.
- Automated deployments and created a modern starter theme with optimised CSS, twig templating, and asset pipeline.
- Conducted performance audits using DevTools and webpagetest.org.
The ASA & British Swimming
Governing bodies for swimming in the UK
Website Solutions Coder
2010 - 2016
- Developed responsive front-end for live events, athlete profiles, and data visualisation charts.
- Developed coaching video platforms using Python, Django, DigitalOcean, AWS S3, and Elastic Transcoder.
Education
Loughborough University
and international exchange at UTS, Sydney
MEng Civil Engineering
2005 - 2009
- Final Year Research Project: Plotting Surveying Data in Google Earth
Java application for conversion between coordinate reference systems using a Helmert datum transformation.
Get in touch
Download my CV, or get in touch at dale@dalestillman.com.