mlos_bench.optimizers.manual_optimizer ====================================== .. py:module:: mlos_bench.optimizers.manual_optimizer .. autoapi-nested-parse:: Manual config suggester (Optimizer) for mlos_bench that proposes an explicit sequence of configurations. This is useful for testing and validation, as it allows you to run a sequence of configurations in a cyclic fashion. .. rubric:: Examples >>> # Load tunables from a JSON string. >>> # Note: normally these would be automatically loaded from the Environment(s)'s >>> # `include_tunables` config parameter. >>> # >>> import json5 as json >>> from mlos_bench.environments.status import Status >>> from mlos_bench.services.config_persistence import ConfigPersistenceService >>> service = ConfigPersistenceService() >>> json_config = ''' ... { ... "group_1": { ... "cost": 1, ... "params": { ... "colors": { ... "type": "categorical", ... "values": ["red", "blue", "green"], ... "default": "green", ... }, ... "int_param": { ... "type": "int", ... "range": [1, 3], ... "default": 2, ... }, ... "float_param": { ... "type": "float", ... "range": [0, 1], ... "default": 0.5, ... // Quantize the range into 3 bins ... "quantization_bins": 3, ... } ... } ... } ... } ... ''' >>> tunables = service.load_tunables(jsons=[json_config]) >>> # Check the defaults: >>> tunables.get_param_values() {'colors': 'green', 'int_param': 2, 'float_param': 0.5} >>> # Now create a ManualOptimizer from a JSON config string. >>> optimizer_json_config = ''' ... { ... "class": "mlos_bench.optimizers.manual_optimizer.ManualOptimizer", ... "description": "ManualOptimizer", ... "config": { ... "max_cycles": 3, ... "tunable_values_cycle": [ ... {"colors": "red", "int_param": 1, "float_param": 0.0}, ... {"colors": "blue", "int_param": 3, "float_param": 1.0}, ... // special case: {} - represents the defaults, without ... // having to copy them from the tunables JSON ... // (which is presumably specified elsewhere) ... {}, ... ], ... } ... } ... ''' >>> config = json.loads(optimizer_json_config) >>> optimizer = service.build_optimizer( ... tunables=tunables, ... service=service, ... config=config, ... ) >>> # Run the optimizer. >>> # Note that the cycle will repeat 3 times, as specified in the config. >>> while optimizer.not_converged(): ... suggestion = optimizer.suggest() ... print(suggestion.get_param_values()) {'colors': 'red', 'int_param': 1, 'float_param': 0.0} {'colors': 'blue', 'int_param': 3, 'float_param': 1.0} {'colors': 'green', 'int_param': 2, 'float_param': 0.5} {'colors': 'red', 'int_param': 1, 'float_param': 0.0} {'colors': 'blue', 'int_param': 3, 'float_param': 1.0} {'colors': 'green', 'int_param': 2, 'float_param': 0.5} {'colors': 'red', 'int_param': 1, 'float_param': 0.0} {'colors': 'blue', 'int_param': 3, 'float_param': 1.0} {'colors': 'green', 'int_param': 2, 'float_param': 0.5} Classes ------- .. autoapisummary:: mlos_bench.optimizers.manual_optimizer.ManualOptimizer Module Contents --------------- .. py:class:: ManualOptimizer(tunables: mlos_bench.tunables.tunable_groups.TunableGroups, config: dict, global_config: dict | None = None, service: mlos_bench.services.base_service.Service | None = None) Bases: :py:obj:`mlos_bench.optimizers.mock_optimizer.MockOptimizer` Optimizer that proposes an explicit sequence of tunable values. Create a new optimizer for the given configuration space defined by the tunables. :param tunables: The tunables to optimize. :type tunables: TunableGroups :param config: Free-format key/value pairs of configuration parameters to pass to the optimizer. :type config: dict :param global_config: :type global_config: dict | None :param service: :type service: Service | None .. py:method:: suggest() -> mlos_bench.tunables.tunable_groups.TunableGroups Always produce the same sequence of explicit suggestions, in a cycle. .. py:property:: supports_preload :type: bool Return True if the optimizer supports pre-loading the data from previous experiments.