Configuration
To run the experiment, the user should define 3 components: the domain of the problem, objectives and hyperparameters of the algorithm. Since it is easiest way to define objective in a python script, GEFEST provides two options for experiments configuration: python config and mixed config.
Python configuration, as the name implies, is just python script, in which user shoud define task objectives, Domain and OptimizationParams objects (TunerParams also if required), and some domain pre-computaions.
Mixed configuration assumes that configuration objects (Domain, OptimizationParams, TunerParams) will be read from yaml file, while objective stay in python script.
OptimizationParams
- pydantic model gefest.core.configs.optimization_params.OptimizationParams[source]
Bases:
BaseModelGEFEST configuration dataclass.
- Variables:
crossover_each_prob (Optional[list[float]]) – Probability for each crossover operation.
Show JSON schema
{ "title": "OptimizationParams", "type": "object", "properties": { "n_steps": { "title": "N Steps", "type": "integer" }, "pop_size": { "title": "Pop Size", "type": "integer" }, "domain": { "$ref": "#/$defs/Domain" }, "objectives": { "items": { "anyOf": [] }, "title": "Objectives", "type": "array" }, "mutations": { "items": { "$ref": "#/$defs/ValidMutations" }, "title": "Mutations", "type": "array" }, "crossovers": { "items": { "$ref": "#/$defs/ValidCrossovers" }, "title": "Crossovers", "type": "array" }, "selector": { "allOf": [ { "$ref": "#/$defs/ValidSelection" } ], "title": "Selector" }, "multiobjective_selector": { "allOf": [ { "$ref": "#/$defs/ValidMultiObjectiveSelection" } ], "default": "moead", "title": "Multiobjective Selector" }, "moead_multi_objective_selector_neighbors": { "default": 2, "title": "Moead Multi Objective Selector Neighbors", "type": "integer" }, "optimizer": { "allOf": [ { "$ref": "#/$defs/ValidOptimizer" } ], "default": "gefest_ga" }, "pair_selector": { "default": null, "title": "Pair Selector" }, "sampler": { "default": null, "title": "Sampler" }, "postprocessor": { "default": null, "title": "Postprocessor" }, "postprocess_rules": { "anyOf": [ { "items": { "anyOf": [] }, "type": "array" }, { "items": { "$ref": "#/$defs/ValidRules" }, "type": "array" } ], "title": "Postprocess Rules" }, "mutation_strategy": { "default": "MutationStrategy", "title": "Mutation Strategy", "type": "string" }, "crossover_strategy": { "default": "CrossoverStrategy", "title": "Crossover Strategy", "type": "string" }, "postprocess_attempts": { "default": 3, "title": "Postprocess Attempts", "type": "integer" }, "mutation_prob": { "default": 0.6, "title": "Mutation Prob", "type": "number" }, "crossover_prob": { "default": 0.6, "title": "Crossover Prob", "type": "number" }, "mutation_each_prob": { "anyOf": [ { "items": { "type": "number" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Mutation Each Prob" }, "crossover_each_prob": { "anyOf": [ { "items": { "type": "number" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Crossover Each Prob" }, "extra": { "default": 5, "title": "Extra", "type": "integer" }, "estimation_n_jobs": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": 1, "title": "Estimation N Jobs" }, "n_jobs": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": -1, "title": "N Jobs" }, "early_stopping_timeout": { "default": 60, "title": "Early Stopping Timeout", "type": "integer" }, "early_stopping_iterations": { "default": 1000, "title": "Early Stopping Iterations", "type": "integer" }, "golem_adapter": { "default": null, "title": "Golem Adapter" }, "tuner_cfg": { "anyOf": [ { "$ref": "#/$defs/TunerParams" }, { "type": "null" } ], "default": null }, "log_dir": { "default": "logs", "title": "Log Dir", "type": "string" }, "run_name": { "default": "run_name", "title": "Run Name", "type": "string" }, "log_dispatcher": { "default": null, "title": "Log Dispatcher" }, "golem_keep_histoy": { "default": false, "title": "Golem Keep Histoy", "type": "boolean" }, "golem_genetic_scheme_type": { "allOf": [ { "$ref": "#/$defs/GeneticSchemeTypesEnum" } ], "default": "steady_state" }, "golem_adaptive_mutation_type": { "allOf": [ { "$ref": "#/$defs/MutationAgentTypeEnum" } ], "default": "default" }, "golem_selection_type": { "allOf": [ { "$ref": "#/$defs/SelectionTypesEnum" } ], "default": "spea2" }, "golem_surrogate_each_n_gen": { "default": 5, "title": "Golem Surrogate Each N Gen", "type": "integer" } }, "$defs": { "Domain": { "description": "Domain configuration dataclass.", "properties": { "allowed_area": { "anyOf": [ { "$ref": "#/$defs/Polygon" }, { "items": { "items": { "type": "number" }, "type": "array" }, "type": "array" } ], "title": "Allowed Area" }, "name": { "default": "main", "title": "Name", "type": "string" }, "min_poly_num": { "default": 2, "title": "Min Poly Num", "type": "integer" }, "max_poly_num": { "default": 4, "title": "Max Poly Num", "type": "integer" }, "min_points_num": { "default": 20, "title": "Min Points Num", "type": "integer" }, "max_points_num": { "default": 50, "title": "Max Points Num", "type": "integer" }, "polygon_side": { "default": 0.0001, "title": "Polygon Side", "type": "number" }, "min_dist_from_boundary": { "default": 0.0001, "title": "Min Dist From Boundary", "type": "number" }, "prohibited_area": { "anyOf": [ { "$ref": "#/$defs/Structure" }, { "type": "string" }, { "type": "null" } ], "default": { "extra_characteristics": {}, "fitness": [], "id_": "60a66964-2c19-42d3-8359-0ff3755c9eb5", "polygons": [] }, "title": "Prohibited Area" }, "fixed_points": { "anyOf": [ { "$ref": "#/$defs/Polygon" }, { "items": { "items": { "type": "number" }, "type": "array" }, "type": "array" }, { "type": "null" } ], "title": "Fixed Points" }, "geometry_is_convex": { "default": true, "title": "Geometry Is Convex", "type": "boolean" }, "geometry_is_closed": { "default": true, "title": "Geometry Is Closed", "type": "boolean" }, "geometry": { "anyOf": [ { "$ref": "#/$defs/Geometry" }, { "type": "string" }, { "type": "null" } ], "default": "2D", "title": "Geometry" } }, "required": [ "allowed_area" ], "title": "Domain", "type": "object" }, "GeneticSchemeTypesEnum": { "enum": [ "steady_state", "generational", "parameter_free" ], "title": "GeneticSchemeTypesEnum", "type": "string" }, "Geometry": { "description": "Abstract geometry class.\n\n\u0421lass contains basic transformations of geometries, geometry properties.\nEach of the methods is overridden for a particular dimension of the geometry.", "properties": {}, "title": "Geometry", "type": "object" }, "MutationAgentTypeEnum": { "enum": [ "default", "random", "bandit", "contextual_bandit", "neural_bandit" ], "title": "MutationAgentTypeEnum", "type": "string" }, "Point": { "description": "2D point dataclass.", "properties": { "x": { "title": "X", "type": "number" }, "y": { "title": "Y", "type": "number" } }, "required": [ "x", "y" ], "title": "Point", "type": "object" }, "PolyID": { "description": "Enumeration of special polygons ids.", "enum": [ "tmp", "constraint", "fixed_area", "fixed_poly", "prohibited_area", "prohibited_target", "prohibited_poly" ], "title": "PolyID", "type": "string" }, "Polygon": { "description": "Polygon dataclass.", "properties": { "points": { "items": { "$ref": "#/$defs/Point" }, "title": "Points", "type": "array" }, "id_": { "anyOf": [ { "format": "uuid", "type": "string" }, { "$ref": "#/$defs/PolyID" }, { "type": "null" } ], "title": "Id " } }, "title": "Polygon", "type": "object" }, "SelectionTypesEnum": { "enum": [ "tournament", "spea2" ], "title": "SelectionTypesEnum", "type": "string" }, "Structure": { "description": "Structure dataclass.", "properties": { "polygons": { "items": { "$ref": "#/$defs/Polygon" }, "title": "Polygons", "type": "array" }, "fitness": { "items": { "type": "number" }, "title": "Fitness", "type": "array" }, "extra_characteristics": { "title": "Extra Characteristics", "type": "object" }, "id_": { "format": "uuid", "title": "Id ", "type": "string" } }, "title": "Structure", "type": "object" }, "TunerParams": { "description": "Dataclass for GolemTuner parameters aggreagtion.\n\nProvides easy configuration for tuner\nwith built-in validation and serialization.", "properties": { "tuner_type": { "title": "Tuner Type", "type": "string" }, "n_steps_tune": { "title": "N Steps Tune", "type": "integer" }, "tune_n_best": { "default": 1, "title": "Tune N Best", "type": "integer" }, "hyperopt_dist": { "title": "Hyperopt Dist", "type": "string" }, "verbose": { "default": true, "title": "Verbose", "type": "boolean" }, "variacne_generator": { "title": "Variacne Generator", "type": "string" }, "timeout_minutes": { "default": 60, "title": "Timeout Minutes", "type": "integer" } }, "required": [ "tuner_type", "n_steps_tune" ], "title": "TunerParams", "type": "object" }, "ValidCrossovers": { "enum": [ "structure_level", "polygon_level" ], "title": "ValidCrossovers", "type": "string" }, "ValidMultiObjectiveSelection": { "enum": [ "moead", "spea2" ], "title": "ValidMultiObjectiveSelection", "type": "string" }, "ValidMutations": { "enum": [ "rotate_poly", "resize_poly", "add_point", "drop_point", "add_poly", "drop_poly", "pos_change_point" ], "title": "ValidMutations", "type": "string" }, "ValidOptimizer": { "enum": [ "gefest_ga", "golem_optimizer", "golem_surrogate" ], "title": "ValidOptimizer", "type": "string" }, "ValidRules": { "enum": [ "not_too_close_polygons", "valid_polygon_geom", "not_out_of_bounds", "not_self_intersects", "not_overlaps_prohibited", "not_too_close_points" ], "title": "ValidRules", "type": "string" }, "ValidSelection": { "enum": [ "roulette_selection", "tournament_selection" ], "title": "ValidSelection", "type": "string" } }, "required": [ "n_steps", "pop_size", "domain", "objectives", "mutations", "crossovers", "selector", "postprocess_rules" ] }
- Config:
arbitrary_types_allowed: bool = True
- Fields:
golem_adaptive_mutation_type (golem.core.optimisers.adaptive.operator_agent.MutationAgentTypeEnum)golem_selection_type (golem.core.optimisers.genetic.operators.selection.SelectionTypesEnum)objectives (list[gefest.core.opt.objective.objective.Objective | Callable])optimizer (gefest.core.configs.optimization_params.ValidOptimizer)selector (Callable | gefest.core.configs.optimization_params.ValidSelection)tuner_cfg (gefest.core.configs.tuner_params.TunerParams | None)
- Validators:
create_classes_instances»all fields
- field n_steps: int [Required]
Number optimization algorithm steps.
- Validated by:
- field pop_size: int [Required]
Popultion size limit. Also auto-generated initial population size.
- Validated by:
- field mutations: list[Callable[[Any], Structure]] | list[ValidMutations] [Required]
Mutation operations.
- Validated by:
- field crossovers: list[Callable[[Any], tuple[Structure]]] | list[ValidCrossovers] [Required]
Crossover operations.
- Validated by:
- field selector: Callable | ValidSelection [Required]
Selection operation.
- Validated by:
- field multiobjective_selector: Callable | ValidMultiObjectiveSelection = 'moead'
Evaluates one-dimensional fitness for multi objective. Uses selector to select individuals by the produces values.
- Validated by:
- field moead_multi_objective_selector_neighbors: int = 2
Parameter used in moead selector.
- Validated by:
- field optimizer: ValidOptimizer = 'gefest_ga'
Optimizer.
- Validated by:
- field pair_selector: Callable = <function panmixis>
Pairs selector for crossover operation.
- Validated by:
- field sampler: Callable = <class 'gefest.tools.samplers.standard.standard.StandardSampler'>
Structures sampler.
- Validated by:
- field postprocessor: Callable = <function Postrocessor.apply_postprocess>
Postprocessing module.
- Validated by:
- field postprocess_rules: list[PolygonRule | StructureRule] | list[ValidRules] [Required]
Postprocessing rules.
- Validated by:
- field mutation_strategy: str = 'MutationStrategy'
Incapsulates mutation step logic.
- Validated by:
- field crossover_strategy: str = 'CrossoverStrategy'
Incapsulates crossover step logic.
- Validated by:
- field postprocess_attempts: int = 3
Nuber of attempts to correct invalid structures on postprocessing.
- Validated by:
- field mutation_prob: float = 0.6
Probability to mutate structure.
- Validated by:
- field crossover_prob: float = 0.6
Probability to crossover pair.
- Validated by:
- field mutation_each_prob: list[float] | None = None
Probability of each provided mutation operation to be applied.
- Validated by:
- field crossover_each_prob: list[float] | None = None
Probability of each provided crossover operation to be applied.
- Validated by:
- field extra: int = 5
Number of extra structures to generate on each optimization step.
- Validated by:
- field estimation_n_jobs: int | None = 1
Number of cores for estimator parallel execution. Use more than 1 core only if you are sure that a particular estimator supports it! For example, if the estimator uses an external simulator, uses parallel calculations by itself, then increasing the number of jobs in the code may not have an effect or even slow down the execution of the program.
- Validated by:
- field n_jobs: int | None = -1
Nuber of cores to use in parallel execution of reproduction operations in GEFEST. n_jobs = -1 to use all cores, n_jobs > 0 to spicic number of cores, n_jobs = 0 to eval sequentially without joblib, use 0 for debug GEFEST only.
- Validated by:
- field early_stopping_timeout: int = 60
GOLEM argument. For details see https://thegolem.readthedocs.io/en/latest/api/parameters.html
- Validated by:
- field early_stopping_iterations: int = 1000
GOLEM argument. For details see:
- Validated by:
- field golem_adapter: Callable = <class 'gefest.core.opt.adapters.structure.StructureAdapter'>
Adapter for convertation GEFEST Structure into GOLEM OptGraph.
- Validated by:
- field tuner_cfg: TunerParams | None = None
Configuration for GOLEM tuner wrap.
- Validated by:
- field log_dir: str = 'logs'
Directory for logs.
- Validated by:
- field run_name: str = 'run_name'
Experiment name.
- Validated by:
- field log_dispatcher: Callable = <class 'gefest.core.utils.logger.LogDispatcher'>
GEFEST logger.
- Validated by:
- field golem_keep_histoy: bool = False
Enables/disables GOLEM experiment historry serialization.
- Validated by:
- field golem_genetic_scheme_type: GeneticSchemeTypesEnum = 'steady_state'
GOLEM configuration argument genetic_scheme_type. For details see:
- Validated by:
- field golem_adaptive_mutation_type: MutationAgentTypeEnum = 'default'
GOLEM configuration argument adaptive_mutation_type. For details see:
- Validated by:
- field golem_selection_type: SelectionTypesEnum = 'spea2'
GOLEM configuration argument selection_type. For details see:
- Validated by:
- field golem_surrogate_each_n_gen: int = 5
Frequency of usage surrogate model in golem_surrogate optimizer
- Validated by:
TunerParams
- pydantic model gefest.core.configs.tuner_params.TunerParams[source]
Bases:
BaseModelDataclass for GolemTuner parameters aggreagtion.
Provides easy configuration for tuner with built-in validation and serialization.
Show JSON schema
{ "title": "TunerParams", "description": "Dataclass for GolemTuner parameters aggreagtion.\n\nProvides easy configuration for tuner\nwith built-in validation and serialization.", "type": "object", "properties": { "tuner_type": { "title": "Tuner Type", "type": "string" }, "n_steps_tune": { "title": "N Steps Tune", "type": "integer" }, "tune_n_best": { "default": 1, "title": "Tune N Best", "type": "integer" }, "hyperopt_dist": { "title": "Hyperopt Dist", "type": "string" }, "verbose": { "default": true, "title": "Verbose", "type": "boolean" }, "variacne_generator": { "title": "Variacne Generator", "type": "string" }, "timeout_minutes": { "default": 60, "title": "Timeout Minutes", "type": "integer" } }, "required": [ "tuner_type", "n_steps_tune" ] }
- Config:
arbitrary_types_allowed: bool = True
- Fields:
- Validators:
- field tuner_type: str [Required]
Type of GOLEM tuners to use. Available: ‘iopt’, ‘optuna’, ‘sequential’, ‘simulataneous’. For tuner details see:
- Validated by:
- field n_steps_tune: int [Required]
Number of tuner steps.
- field tune_n_best: int = 1
Top tune_n_best structures from provided population to tune.
- field hyperopt_dist: Callable | str = <function hp_uniform>
Random distribution function.
- Validated by:
- field verbose: bool = True
GOLEM console info.
- field variacne_generator: Callable[[Structure], list[float]] | str = <function percent_edge_variance>
The function for generating the search space includes intervals for each component of each point of each polygon in the provided structure.
- Output format should be spicific dict configuration. For details see:
- Validated by:
- field timeout_minutes: int = 60
GOLEM argument.
- validator tuner_type_validate » tuner_type[source]
Checks if specified tuner exists.
- validator hyperopt_fun_validate » hyperopt_dist[source]
Checks if hyperopt distribution function exists.
- validator variacne_generator_fun_validate » variacne_generator[source]
Checks if specified variance generation function exists.
Domain
- pydantic model gefest.core.geometry.domain.Domain[source]
Bases:
BaseModelDomain configuration dataclass.
Show JSON schema
{ "title": "Domain", "description": "Domain configuration dataclass.", "type": "object", "properties": { "allowed_area": { "anyOf": [ { "$ref": "#/$defs/Polygon" }, { "items": { "items": { "type": "number" }, "type": "array" }, "type": "array" } ], "title": "Allowed Area" }, "name": { "default": "main", "title": "Name", "type": "string" }, "min_poly_num": { "default": 2, "title": "Min Poly Num", "type": "integer" }, "max_poly_num": { "default": 4, "title": "Max Poly Num", "type": "integer" }, "min_points_num": { "default": 20, "title": "Min Points Num", "type": "integer" }, "max_points_num": { "default": 50, "title": "Max Points Num", "type": "integer" }, "polygon_side": { "default": 0.0001, "title": "Polygon Side", "type": "number" }, "min_dist_from_boundary": { "default": 0.0001, "title": "Min Dist From Boundary", "type": "number" }, "prohibited_area": { "anyOf": [ { "$ref": "#/$defs/Structure" }, { "type": "string" }, { "type": "null" } ], "default": { "extra_characteristics": {}, "fitness": [], "id_": "60a66964-2c19-42d3-8359-0ff3755c9eb5", "polygons": [] }, "title": "Prohibited Area" }, "fixed_points": { "anyOf": [ { "$ref": "#/$defs/Polygon" }, { "items": { "items": { "type": "number" }, "type": "array" }, "type": "array" }, { "type": "null" } ], "title": "Fixed Points" }, "geometry_is_convex": { "default": true, "title": "Geometry Is Convex", "type": "boolean" }, "geometry_is_closed": { "default": true, "title": "Geometry Is Closed", "type": "boolean" }, "geometry": { "anyOf": [ { "$ref": "#/$defs/Geometry" }, { "type": "string" }, { "type": "null" } ], "default": "2D", "title": "Geometry" } }, "$defs": { "Geometry": { "description": "Abstract geometry class.\n\n\u0421lass contains basic transformations of geometries, geometry properties.\nEach of the methods is overridden for a particular dimension of the geometry.", "properties": {}, "title": "Geometry", "type": "object" }, "Point": { "description": "2D point dataclass.", "properties": { "x": { "title": "X", "type": "number" }, "y": { "title": "Y", "type": "number" } }, "required": [ "x", "y" ], "title": "Point", "type": "object" }, "PolyID": { "description": "Enumeration of special polygons ids.", "enum": [ "tmp", "constraint", "fixed_area", "fixed_poly", "prohibited_area", "prohibited_target", "prohibited_poly" ], "title": "PolyID", "type": "string" }, "Polygon": { "description": "Polygon dataclass.", "properties": { "points": { "items": { "$ref": "#/$defs/Point" }, "title": "Points", "type": "array" }, "id_": { "anyOf": [ { "format": "uuid", "type": "string" }, { "$ref": "#/$defs/PolyID" }, { "type": "null" } ], "title": "Id " } }, "title": "Polygon", "type": "object" }, "Structure": { "description": "Structure dataclass.", "properties": { "polygons": { "items": { "$ref": "#/$defs/Polygon" }, "title": "Polygons", "type": "array" }, "fitness": { "items": { "type": "number" }, "title": "Fitness", "type": "array" }, "extra_characteristics": { "title": "Extra Characteristics", "type": "object" }, "id_": { "format": "uuid", "title": "Id ", "type": "string" } }, "title": "Structure", "type": "object" } }, "required": [ "allowed_area" ] }
- Fields:
allowed_area (gefest.core.geometry.datastructs.polygon.Polygon | list[list[float]])fixed_points (gefest.core.geometry.datastructs.polygon.Polygon | list[list[float]] | None)geometry (gefest.core.geometry.geometry.Geometry | str | None)geometry_is_closed (bool)geometry_is_convex (bool)max_points_num (int)max_poly_num (int)min_dist_from_boundary (float)min_points_num (int)min_poly_num (int)name (str)polygon_side (float)prohibited_area (gefest.core.geometry.datastructs.structure.Structure | str | None)
- Validators:
_post_init_validation»all fieldsvalidate_allowed_area»allowed_areavalidate_fixed_points»fixed_pointsvalidate_max_poly_num»max_poly_numvalidate_min_points_num»min_points_numvalidate_min_poly_num»min_poly_numvalidate_prohibited_area»prohibited_area
- property dist_between_polygons: float
Min distance between polygons instructure.
- property dist_between_points: float
Min dstance between neighbours points in polygon.
- property min_x: int
Min x domain coord.
- property max_x: int
Max x domain coord.
- property min_y: int
Min y domain coord.
- property max_y: int
Max y domain coord.
- property len_x: int
Len of x domain side.
- property len_y: int
Len of y domain side.
Utils
- gefest.core.configs.utils.load_config(cfg_py_path: str, cfg_yaml_path: str | None = None, *args, **kwargs) OptimizationParams[source]
Generates configuretion files from yaml files.
- Parameters:
cfg_py_path (str) – Path to metrics.py.
cfg_yaml_path (str) – Path to config.yaml.
- Returns:
GEFEST unified configuretion file.
- Return type:
- gefest.core.configs.utils.create_prohibited(points_radius, targets: list[list] | None = None, fixed_points: list[list] | None = None, fixed_area: list[list] | None = None) Structure[source]
Creates of fixed, prohibited structures. Polygons cannot cross them.
- Parameters:
points_radius (_type_) – _description_
targets (Optional[list[list]], optional) – Fixed targets inside domain. Defaults to None.
fixed_points (Optional[list[list]], optional) – Fixed lines inside domain. Defaults to None.
fixed_area (Optional[list[list]], optional) – Fixed areas inside domain. Defaults to None.
- Returns:
Compilation of inputs as GEFEST Structure.
- Return type: