Achieve the optimal ML model in an automated and smart way

Photo by Alexis Baydoun on Unsplash

Table of contents

  1. Hyperparameters and scikit-learn Tunning Methods
  2. Tunning Hyperparameters with Optuna
  3. Conclusion


I recently finished my first machine learning project on Kaggle, predicting sale price with the Boston Housing dataset (co-author: Julia Yang). Please find our work, which achieved top 9% among 5k competitors, at [Top 9%] House Price.

From this project, Julia and I both learnt a lot, and we will write up a series of articles about our learnings. In this one, I would like to discuss my experience with tuning hyperparameters of ML models using Optuna.

Hyperparameters and scikit-learn Tunning Methods

In a machine learning project, tuning the hyperparameters is one of the…

Recently, I’m working on creating a MySQL RDS instance with CloudFormation template. Although there have been tons of documents from both AWS officials or the community talking about this, I feel there are not many documents diving deep about making MasterUserPassword secure.

When creating a MySQL instance, RDS will create a “master user” with all the permissions needed for managing the database. Disregarding the CloudFormation reference saying MasterUsername and MasterUserPassword are not required, they are required to create a MySQL RDS instance. You will get an error “Property MasterUserPassword cannot be empty” if they’re not provided in the template. It…

Image from

If you’re not familiar with AWS Step Functions, it is a service where we can create state-machines to orchestrate asynchronous tasks in workflows without worrying too much about any underlying infrastructure and operations. I’ve been using it a lot recently, and I’d like to share the way I implement a finally block in a Step Functions state machine.

In a workflow, there could be some operations that need to be executed no matter what happens, for example, cleaning up temporary resources that are created at the beginning of the workflow. …

This machine is a server! (London Science Museum, by Binary Koala)

I’ve been using the Java library dnsjava(org.xbill.DNS) a lot recently to make DNS queries. Since my work will rely on the behavior of that library, I figured I have to understand it better. One way to understand a library will be to read through all the code, but I prefer to write some tests to validate my understanding.

The application code is straightforward as the following. One thing I wanted to understand was how the library would handle retrying when the DNS query fails, for example, timed out.

Application Code

I tried to write some tests by mocking the Resolver…

In Go, append(slice, x) is one of the most used built-in functions. But have you ever thought about a simple question: what is the time complexity of it?

The short answer: The amortized cost of append() is O(1), but the worst-case cost is O(N).

Before I move to the long answer, I’d like to add a disclaimer first: The Go Programming Language Specification is supposed to be a single source of truth. Everything else can be treated as implementation details, which may or may not be consistent across versions, platforms, and compilers. What I’m writing here is all about those…

Let’s start directly with an example (go playground):

package mainimport "fmt"func main() {    var a interface{}
fmt.Printf("a == nil is %t\n", a == nil)
var b interface{}
var p *int = nil
b = p
fmt.Printf("b == nil is %t\n", b == nil)

What is your expectation for the output? It will end up like:

a == nil is true
b == nil is false

It is somehow counter-intuitive at first glance, but it makes a lot more sense if we explore a little bit about the reflection model in Golang.

Under the hood, an interface in…

Gary Lu

Software Developer in AWS.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store