This is going to be another shorter post, I've been pretty unmotivated to work on my bigger long-term projects lately (I think my last commit to any of them was 3 months ago). Maybe next post though...
So around April, I came across a post from Carl McBride Ellis, a data scientist and I guess LinkedIn influencer, about a small Kaggle competition he was running called The Perfect Fit. This immediately caught my attention, as it seemed to be very different from the typical Kaggle competitions in that it was more like a puzzle with the goal of trying to reverse engineer the data generation process rather than the standard throwing the kitchen sink of fancy machine learning models at the problem.
The contest ended a little over two weeks ago as of writing, and I was fortunate enough to (a) be 1 of 8 participants/teams out of 68 total to successfully reverse engineer the problem, (b) actually place 1st overall in terms of mean absolute error, and (c) win a free copy of The Orange Book of Machine Learning (Green Edition) based on my solution write-up.
I will attempt to briefly summarize the problem context and my approach at a high level.
The problem involved a small
synthetic tabular dataset with the following columns:
id,
x1, x2, Country, City, x4,
x5, x6, x7, x8, x9, x10,
x11, and target. The goal was to predict the target variable for
a test set.
Through a mix of data exploration, partial dependence analysis, and common sense, you can figure out that
(a) Country, x6, and x7 are completely useless, and (b) for
id >= 100 and x5 != 999 (x5 was contaminated with sentinel
values), target is just a linear combination of the remaining features and a few simple
variables derived from them.
For x5 != 999, it turns out that x5 ~ Unif(7, 12), so you can actually brute
force the random seed that was used to generate the data to recover what the original x5
values were before some were replaced with 999. The random seed ended up being 4242, so I
guess Hitchhiker's Deep Thought was onto something after all. The same linear equation applies to the
recovered x5 values, so the problem is solved.
I ended up going a step further and brute forcing every possible parenthesization of the linear combination as a labeled binary expression tree since floating point addition is not associative. Completely unnecessary, but it did shave down some of the MAE resulting from floating point dust.
I'm slowly making my way through the e-book and so far it's been quite good. So I think participating in the contest was pretty worth it.