Getting started¶
In the following it is showed how to setup optimization problems in BoFire and how to use strategies to solve them.
Setting up the optimization problem¶
In BoFire, an optimization problem is defined by defining a domain containing input and output features as well as constraints (optional).
Features¶
Input features can be continuous, discrete, categorical, or categorical with descriptors:
from bofire.data_models.features.api import ContinuousInput, DiscreteInput, CategoricalInput, CategoricalDescriptorInput
x1 = ContinuousInput(key="x1", bounds=(0,1))
x2 = ContinuousInput(key="x2", bounds=(0,1))
x3 = ContinuousInput(key="x3", bounds=(0,1))
x4 = DiscreteInput(key="x4", values=[1, 2, 5, 7.5])
x5 = CategoricalInput(key="x5", categories=["A", "B", "C"], allowed=[True,True,False])
x6 = CategoricalDescriptorInput(key="x6", categories=["c1", "c2", "c3"], descriptors=["d1", "d2"], values = [[1,2],[2,5],[1,7]])
As output features, currently only continuous output features are supported. Each output feature should have an objective, which can be a minimize or maximize objective. Furthermore, we can define weights between 0 and 1 in case the objectives should not be weighted equally.
from bofire.data_models.features.api import ContinuousOutput
from bofire.data_models.objectives.api import MaximizeObjective, MinimizeObjective
objective1 = MaximizeObjective(
w=1.0,
bounds= [0.0,1.0],
)
y1 = ContinuousOutput(key="y1", objective=objective1)
objective2 = MinimizeObjective(
w=1.0
)
y2 = ContinuousOutput(key="y2", objective=objective2)
In- and output features are collected in respective feature lists.
from bofire.data_models.domain.api import Inputs, Outputs
input_features = Inputs(features = [x1, x2, x3, x4, x5, x6])
output_features = Outputs(features=[y1, y2])
A summary of the constraints can be obtained by the method get_reps_df
:
input_features.get_reps_df()
Type | Description | |
---|---|---|
x1 | ContinuousInput | [0.0,1.0] |
x2 | ContinuousInput | [0.0,1.0] |
x3 | ContinuousInput | [0.0,1.0] |
x4 | DiscreteInput | type='DiscreteInput' key='x4' unit=None values... |
x6 | CategoricalDescriptorInput | 3 categories |
x5 | CategoricalInput | 3 categories |
output_features.get_reps_df()
Type | Description | |
---|---|---|
y1 | ContinuousOutput | ContinuousOutputFeature |
y2 | ContinuousOutput | ContinuousOutputFeature |
y3 | ContinuousOutput | ContinuousOutputFeature |
Individual features can be retrieved by name.
x5 = input_features.get_by_key('x5')
x5
CategoricalInput(type='CategoricalInput', key='x5', categories=['A', 'B', 'C'], allowed=[True, True, False])
This is also possible with list of feature names.
input_features.get_by_keys(['x5', 'x2'])
Inputs(type='Inputs', features=[ContinuousInput(type='ContinuousInput', key='x2', unit=None, bounds=(0.0, 1.0), local_relative_bounds=None, stepsize=None), CategoricalInput(type='CategoricalInput', key='x5', categories=['A', 'B', 'C'], allowed=[True, True, False])])
Features of a specific type can be returned by the get
method, by default it returns all features that are an instance of the provided class.
input_features.get(CategoricalInput)
Inputs(type='Inputs', features=[CategoricalDescriptorInput(type='CategoricalDescriptorInput', key='x6', categories=['c1', 'c2', 'c3'], allowed=[True, True, True], descriptors=['d1', 'd2'], values=[[1.0, 2.0], [2.0, 5.0], [1.0, 7.0]]), CategoricalInput(type='CategoricalInput', key='x5', categories=['A', 'B', 'C'], allowed=[True, True, False])])
By using the exact
argument one can force it to only return feature of the exact same class.
input_features.get(CategoricalInput, exact=True)
Inputs(type='Inputs', features=[CategoricalInput(type='CategoricalInput', key='x5', categories=['A', 'B', 'C'], allowed=[True, True, False])])
The get_keys
method follows the same logic as the get
method but returns just the keys of the features instead of the features itself.
input_features.get_keys(CategoricalInput)
['x6', 'x5']
The input feature container further provides methods to return a feature container with only all fixed or all free features.
free_inputs = input_features.get_free()
fixed_inputs = input_features.get_fixed()
One can uniformly sample from individual input features.
x5.sample(2)
0 B 1 A Name: x5, dtype: object
Or directly from input feature containers, uniform, sobol and LHS sampling is possible. A default, uniform sampling is used.
from bofire.data_models.enum import SamplingMethodEnum
X = input_features.sample(n=10, method=SamplingMethodEnum.LHS)
X
x1 | x2 | x3 | x4 | x6 | x5 | |
---|---|---|---|---|---|---|
0 | 0.423139 | 0.305001 | 0.881045 | 2.0 | c3 | A |
1 | 0.873972 | 0.525925 | 0.674935 | 7.5 | c3 | A |
2 | 0.782031 | 0.867259 | 0.442600 | 2.0 | c1 | B |
3 | 0.691130 | 0.403864 | 0.348524 | 7.5 | c3 | B |
4 | 0.051185 | 0.733657 | 0.144178 | 1.0 | c2 | A |
5 | 0.939134 | 0.199665 | 0.226415 | 1.0 | c1 | A |
6 | 0.323216 | 0.912386 | 0.066617 | 1.0 | c1 | B |
7 | 0.280553 | 0.208415 | 0.544485 | 7.5 | c3 | A |
8 | 0.163496 | 0.022924 | 0.707360 | 5.0 | c2 | B |
9 | 0.554554 | 0.673069 | 0.938194 | 5.0 | c1 | B |
Constraints¶
The search space can be further defined by constraints on the input features. BoFire supports linear equality and inequality constraints, as well as non-linear equality and inequality constraints.
Linear constraints¶
LinearEqualityConstraint
and LinearInequalityConstraint
are expressions of the form $\sum_i a_i x_i = b$ or $\leq b$ for equality and inequality constraints respectively.
They take a list of names of the input features they are operating on, a list of left-hand-side coefficients $a_i$ and a right-hand-side constant $b$.
from bofire.data_models.constraints.api import LinearEqualityConstraint, LinearInequalityConstraint
# A mixture: x1 + x2 + x3 = 1
constr1 = LinearEqualityConstraint(features=["x1", "x2", "x3"], coefficients=[1,1,1], rhs=1)
# x1 + 2 * x3 < 0.8
constr2 = LinearInequalityConstraint(features=["x1", "x3"], coefficients=[1, 2], rhs=0.8)
Linear constraints can only operate on ContinuousInput
features.
Nonlinear constraints¶
NonlinearEqualityConstraint
and NonlinearInequalityConstraint
take any expression that can be evaluated by pandas.eval, including mathematical operators such as sin
, exp
, log10
or exponentiation. So far, they cannot be used in any optimizations.
from bofire.data_models.constraints.api import NonlinearEqualityConstraint, NonlinearInequalityConstraint
# The unit circle: x1**2 + x2**2 = 1
const3 = NonlinearEqualityConstraint(expression="x1**2 + x2**2 - 1")
const3
NonlinearEqualityConstraint(type='NonlinearEqualityConstraint', expression='x1**2 + x2**2 - 1', features=None, jacobian_expression=None)
Combinatorial constraint¶
Use NChooseKConstraint
to express that we only want to have $k$ out of the $n$ parameters to take positive values.
Think of a mixture, where we have long list of possible ingredients, but want to limit number of ingredients in any given recipe.
from bofire.data_models.constraints.api import NChooseKConstraint
# Only 2 or 3 out of 3 parameters can be greater than zero
constr5 = NChooseKConstraint(features=["x1", "x2", "x3"], min_count=2, max_count=3, none_also_valid=True)
constr5
NChooseKConstraint(type='NChooseKConstraint', features=['x1', 'x2', 'x3'], min_count=2, max_count=3, none_also_valid=True)
Note that we have to set a boolean, if None is also a valid selection, e.g. if we want to have 2 or 3 or none of the ingredients in our recipe.
Similar to the features, constraints can be grouped in a container which acts as the union constraints.
from bofire.data_models.domain.api import Constraints
constraints = Constraints(constraints=[constr1, constr2])
A summary of the constraints can be obtained by the method get_reps_df
:
constraints.get_reps_df()
Type | Description | |
---|---|---|
0 | LinearEqualityConstraint | type='LinearEqualityConstraint' features=['x1'... |
1 | LinearInequalityConstraint | type='LinearInequalityConstraint' features=['x... |
We can check whether a point satisfies individual constraints or the list of constraints.
constr2.is_fulfilled(X).values
array([False, False, False, False, True, False, True, False, False, False])
Output constraints can be setup via sigmoid-shaped objectives passed as argument to the respective feature, which can then also be plotted.
from bofire.data_models.objectives.api import MinimizeSigmoidObjective
from bofire.plot.api import plot_objective_plotly
output_constraint = MinimizeSigmoidObjective(
w=1.0,
steepness=10,
tp=0.5
)
y3= ContinuousOutput(key="y3", objective=output_constraint)
output_features = Outputs(features=[y1, y2, y3])
fig = plot_objective_plotly(feature=y3, lower=0, upper=1)
fig.show()
/opt/homebrew/Caskroom/miniforge/base/envs/bofire-2/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
The domain¶
The domain holds then all information about an optimization problem and can be understood as a search space defintion.
from bofire.data_models.domain.api import Domain
domain = Domain(
inputs=input_features,
outputs=output_features,
constraints=constraints
)
In addition one can instantiate the domain also just from lists.
domain_single_objective = Domain.from_lists(
inputs=[x1, x2, x3, x4, x5, x6],
outputs=[y1],
constraints=[]
)
Optimization¶
To solve the optimization problem, we further need a solving strategy. BoFire supports strategies without a prediction model such as a random strategy and predictive strategies which are based on a prediction model.
All strategies contain an ask
method returning a defined number of candidate experiments.
Random Strategy¶
from bofire.data_models.strategies.api import RandomStrategy
import bofire.strategies.api as strategies
strategy_data_model = RandomStrategy(domain=domain)
random_strategy = strategies.map(strategy_data_model)
random_candidates = random_strategy.ask(2)
random_candidates
x1 | x2 | x3 | x4 | x6 | x5 | |
---|---|---|---|---|---|---|
0 | 0.516301 | 0.358447 | 0.125253 | 7.5 | c3 | A |
1 | 0.246566 | 0.636906 | 0.116528 | 2.0 | c1 | B |
Single objective Bayesian Optimization strategy¶
Since a predictive strategy includes a prediction model, we need to generate some historical data, which we can afterwards pass as training data to the strategy via the tell method.
For didactic purposes we just choose here from one of our benchmark methods.
from bofire.benchmarks.single import Himmelblau
benchmark = Himmelblau()
(benchmark.domain.inputs + benchmark.domain.outputs).get_reps_df()
Type | Description | |
---|---|---|
x_1 | ContinuousInput | [-6.0,6.0] |
x_2 | ContinuousInput | [-6.0,6.0] |
y | ContinuousOutput | ContinuousOutputFeature |
Generating some initial data works as follows:
samples = benchmark.domain.inputs.sample(10)
experiments = benchmark.f(samples, return_complete=True)
experiments
x_1 | x_2 | y | valid_y | |
---|---|---|---|---|
0 | -5.207328 | 3.267036 | 378.064959 | 1 |
1 | -3.542455 | 5.285482 | 349.256442 | 1 |
2 | -5.155535 | 5.077326 | 612.311571 | 1 |
3 | -5.316850 | 3.642571 | 438.194554 | 1 |
4 | -3.701859 | -5.987050 | 642.945914 | 1 |
5 | -1.165247 | -0.212096 | 163.045785 | 1 |
6 | 3.267629 | 2.292458 | 6.199849 | 1 |
7 | -0.915547 | 1.141966 | 125.068321 | 1 |
8 | -2.672275 | -1.027612 | 98.118896 | 1 |
9 | 5.363115 | -4.279275 | 459.876833 | 1 |
Let's setup the SOBO strategy and ask for a candidate.
from bofire.data_models.strategies.api import SoboStrategy
from bofire.data_models.acquisition_functions.api import qNEI
sobo_strategy_data_model = SoboStrategy(domain=benchmark.domain, acquisition_function=qNEI())
sobo_strategy = strategies.map(sobo_strategy_data_model)
sobo_strategy.tell(experiments=experiments)
sobo_strategy.ask(candidate_count=1)
x_1 | x_2 | y_pred | y_sd | y_des | |
---|---|---|---|---|---|
0 | 2.185807 | 5.14596 | 48.612437 | 208.728779 | -48.612437 |
Design of Experiments¶
As a simple example for the DoE functionalities we consider the task of finding a D-optimal design for a fully-quadratic model with three design variables with bounds (0,1) and a mixture constraint.
We define the design space including the constraint as a domain. Then we pass it to the optimization routine and specify the model. If the user does not indicate a number of experiments it will be chosen automatically based on the number of model terms.
from bofire.strategies.doe.design import find_local_max_ipopt
import numpy as np
domain = Domain(
inputs=[x1,x2,x3],
outputs=[y1],
constraints=[constr1]
)
res = find_local_max_ipopt(domain, "fully-quadratic")
np.round(res,3)
****************************************************************************** This program contains Ipopt, a library for large-scale nonlinear optimization. Ipopt is released as open source code under the Eclipse Public License (EPL). For more information visit https://github.com/coin-or/Ipopt ******************************************************************************
x1 | x2 | x3 | |
---|---|---|---|
exp0 | 0.5 | 0.5 | -0.0 |
exp1 | -0.0 | 1.0 | -0.0 |
exp2 | -0.0 | 0.5 | 0.5 |
exp3 | -0.0 | 0.5 | 0.5 |
exp4 | 0.5 | -0.0 | 0.5 |
exp5 | 0.5 | 0.5 | -0.0 |
exp6 | -0.0 | 1.0 | -0.0 |
exp7 | 1.0 | -0.0 | -0.0 |
exp8 | -0.0 | -0.0 | 1.0 |
exp9 | -0.0 | -0.0 | 1.0 |
exp10 | 0.5 | -0.0 | 0.5 |
exp11 | 0.5 | -0.0 | 0.5 |
exp12 | 0.5 | 0.5 | -0.0 |
The resulting design looks like this:
import matplotlib.pyplot as plt
fig = plt.figure(figsize=((10,10)))
ax = fig.add_subplot(111, projection='3d')
ax.view_init(45, 45)
ax.set_title("fully-quadratic model")
ax.set_xlabel("$x_1$")
ax.set_ylabel("$x_2$")
ax.set_zlabel("$x_3$")
plt.rcParams["figure.figsize"] = (10,8)
#plot feasible polytope
ax.plot(
xs=[1,0,0,1],
ys=[0,1,0,0],
zs=[0,0,1,0],
linewidth=2
)
#plot D-optimal solutions
ax.scatter(xs=res["x1"], ys=res["x2"], zs=res["x3"], marker="o", s=40, color="orange")
<mpl_toolkits.mplot3d.art3d.Path3DCollection at 0x17d571330>