Join Newsletter

Image Classification in a Noisy Fraudulent World - A Journey of Computational and Statistical Performance

YOW! Data 2019

Formbay's fraud detection system relies on classification of photographic evidence to verify solar installations. Over the last 10 years, Formbay has amassed over 10 million labelled images of solar installations. Image classification over Formbay's dataset sounds easy. Lots of data, apply neural networks and profit from automation! However with such a large dataset, there is room for lots of noise. Noise such as mislabelled images, overlapping classes, corrupted image data, imbalanced classes, rotational variance and more.

This presentation demonstrates how we built our Image Processing pipeline tackling these noise issues while addressing class/concept drift. First we'll examine the data-situation of Formbay when we started and our initial model. Then we'll address each statistical and computational problem we met and how we decided to address them, slowly evolving our data pipeline over time.

This presentation focuses on the complexities of engineering production ready ML systems which involve balancing between statistical ("how accurate") and computational performance ("how fast").

Roger Qiu

Principal Machine Learning Engineer Consultant

Matrix AI


Roger Qiu is a Software Entrepreneur/Engineer specialising in cloud infrastructure automation, computer-vision artificial intelligence, geographic information systems and front-end architecture.

He is the founder of Matrix AI (, a Sydney based company that provides end to end machine learning application consulting services, and invented the Matrix Operating System for Cloud, Machine Learning and IoT orchestration.

Roger Qiu is currently the Principal Machine Learning Engineer Consultant leading the data team at Formbay (, the leading SaaS platform for paperless management of Solar Trading Credit applications and document workflow management. Formbay's data team is engaged in projects involving image classification of photographic evidence, object detection of solar panels in aerial and satellite imagery, and document analysis of paper forms.

Roger Qiu loves piloting sailplanes and is a member of the Southern Cross Gliding Glub! ✈✈✈