Abstract

Missing data is a fundamental obstacle in the practice of data science. This paper surveys a few conventions for imputation as available in the Automunge open source python library platform for tabular data preprocessing, including “ML infill” in which auto ML models are trained for target features from partitioned extracts of a training set. A series of validation experiments were performed to benchmark imputation scenarios towards downstream model performance, in which it was found for the given benchmark sets that in many cases ML infill outperformed for both numeric and categoric target features, and was otherwise at minimum within noise…

Introduction

With the annual ICLR conference right around the corner, have decided this year to reconsider my approach for explorations. You see there has become somewhat of an explosion of papers accepted at these top tier machine learning research venues, and well finally came to the conclusion that if you don’t approach with an objective in mind an attendee is likely to get somewhat overwhelmed amongst the forest of signal and noise. So what will be my reward function for this year’s ICLR? Trying to get caught up in the specialty of reinforcement learning.

Of course I’ve had a few brush-ins…

Introduction

As the Automunge project closes in on two years of full time focus, figured it would be worth a little reflection on the surrounding competitive environment, which has gone through some considerable changes in the time since inception. Automunge was conceived as a platform for tabular data preprocessing, basically with the intent to automate most if not all of the data science workflow immediately preceding the application of machine learning.

We made an early decision to build on top of the Pandas dataframe library, which for our purposes has served quite well, however this library does have some built in…

Small update to appendix A associated with parameter consolidations in 5.94

https://twitter.com/_NicT_/status/1382464994785234948?s=20

The following essay was directly inspired by the reporting of Eric Berger in his recently published book Liftoff, documenting the early days of SpaceX. A link to purchase is provided here and again at the conclusion.

Liftoff — Eric Berger

Just finished reading a really impressive book on the fundamentals of quantum computing. In my experience many books that cover this territory get somewhat lost in the formality of quantum notations and linear algebra formulations without imparting any real intuition on the mechanics behind quantum algorithms. This book, Programming Quantum Computers by Eric Johnston, Nic Hurrigan, and Mercedes Gimeno-Segovia, turned out to be the single most helpful book I’ve come across for clearly articulating what is taking place in fundamental algorithms like Grover’s search and Shor’s factoring algorithms. …

Abstract

Missing data is a fundamental obstacle in the practice of data science. This paper surveys a few conventions for imputation as available in the Automunge open source python library platform for tabular data preprocessing, including ML infill in which auto ML models are trained for target features from partitioned extracts of a training set. A series of validation experiments were performed to benchmark imputation scenarios towards downstream model performance, in which it was found for the given benchmark sets that ML infill performed best for numeric target columns in cases of missing not at random, and was otherwise at minimum…