Overview

The Black Box Optimization Challenge is a competition hosted by NeurIPS 2020. The challenge focuses on the application of Bayesian optimization to tune the hyperparameters of machine learning models. Participants are required to submit their optimization algorithms and compete on a leaderboard based on the performance of their algorithms on held-out objective functions. Local experimentation and benchmarking can be done using the Bayesmark package.

Features

Starter kit for the black box optimization challenge at NeurIPS 2020
Upload submissions for the challenge
Benchmark site powered by Valohai and runs the Bayesmark package
Surrogate model (often a Gaussian process) for the objective function
Acquisition function to determine the most promising point to evaluate next
Application of Bayesian optimization to hyperparameter tuning of machine learning models
Local experimentation and benchmarking using Bayesmark package
Quick-start instructions for submissions on the challenge website

Summary

The Black Box Optimization Challenge at NeurIPS 2020 is a competition focused on the application of Bayesian optimization to tune the hyperparameters of machine learning models. Participants can upload their optimization algorithms and compete on a leaderboard. The challenge provides a starter kit and a benchmark site powered by Valohai and the Bayesmark package for evaluation. Local experimentation and benchmarking can be done using the Bayesmark package, with a provided script for convenience. The challenge aims to compare different approaches to Bayesian optimization across a large number of problems.

Rdturnermtl

Bbo_challenge_starter_kit