pycomex package

Subpackages

• pycomex.app package
  • Module contents

Submodules

pycomex.cli module

Console script for pycomex.

class pycomex.cli.ExperimentCLI(name: str, experiments_path: str, experiments_base_path: Optional[str] = None, version: str = '0.0.0', additional_help: str = '', **kwargs)

Bases: Group

callback(version)

discover_experiments()

experiment_info = <Command info>

list_experiments = <Command list>

run_experiment = <Command run>

pycomex.cli.section(string: str, length: int, padding: int = 2)

pycomex.cli.subsection(string: str, length: int, padding: int = 2)

pycomex.experiment module

This is a module level docstring

Header

is this ok?

class pycomex.experiment.ArchivedExperiment(archive_path: str)

Bases: object

import_experiment_module()

class pycomex.experiment.Experiment(base_path: str, namespace: str, glob: dict, debug_mode: bool = False, templates: ~typing.Dict[str, ~jinja2.environment.Template] = {'analysis.py': <Template 'analysis.py.j2'>, 'annotations.rst': <Template 'annotations.py.j2'>}, work_tracker_class: ~typing.Type[~pycomex.work.AbstractWorkTracker] = <class 'pycomex.work.NaiveWorkTracker'>)

Bases: object

Context Manager to wrap the main business logic of a computational experiment.

DATETIME_FORMAT = '%A, %d %b %Y  at %H:%M'

DEFAULT_TEMPLATES = {'analysis.py': <Template 'analysis.py.j2'>, 'annotations.rst': <Template 'annotations.py.j2'>}

commit_fig(name: str, fig: object, bbox_inches='tight', pad_inches=0.05) → None

commit_json(name: str, data: dict, indent: int = 4) → None

commit_raw(name: str, content: str) → None

copy_source() → None

determine_path() → str

discover_parameters() → None

property end_time_pretty: str

info(message: str) → None

Logs the given message string as an “INFO” level log message.

Parameters

message (str) – The string message to be printed as a log.

Returns

None

info_lines(message: str) → None

Logs each line of the given multiline message string as an “INFO” level log message.

Parameters

message (str) – The string message to be printed as a log

Returns

None

load_parameters_json(content: str) → None

load_parameters_py(content: str) → None

load_records() → None

open(name: str, mode: str = 'w')

prepare_logger() → None

Creates the internal logger instance and registers all the appropriate handlers.

Before this method logger is None. Only afterwards the logger is actually initialized. Afterwards all the necessary handlers are also attached. This method omits the file handler if the experiment is imported instead of executed.

prepare_path() → None

read_parameters(path: str) → None

render_template(file_name)

render_templates()

save_experiment_data() → None

Saves the internal experiment data dictionary as a JSON file with the already pre-determined path self.data_path.

Returns

None

save_experiment_error(exception_value, exception_traceback) → None

save_experiment_meta() → None

Saves the internal experiment metadata dictionary into as a json file with the already pre-determined path self.meta_path

Returns

None

property start_time_pretty: str

status(log: bool = True) → None

This function primarily updates the metadata of the experiment such as the total runtime up to that point as well as some hardware information such as the CPU and RAM usage.

Optionally also prints this information as an INFO type log message

Returns

None

update(n: int = 1, weight: float = 1.0, monitor_resources=True)

update_monitoring() → dict

property work: int

write_path(path: str) → None

class pycomex.experiment.ExperimentArgParser(name: str, path: str, description: str)

Bases: ArgumentParser

static valid_params(value: str) → str

class pycomex.experiment.ExperimentRegistry(base_path: str, max_depth: int = 5)

Bases: object

load()

traverse_folder(path_elements: List[str], recursion_depth: int = 0)

class pycomex.experiment.NamespaceFolder(folder_path: str)

Bases: object

class MetaPlaceholder

Bases: object

meta_getitem(key)

update()

exception pycomex.experiment.NoExperimentExecution

Bases: Exception

class pycomex.experiment.PrintDescriptionAction(*args, description, **kwargs)

Bases: Action

pycomex.experiment.run_example(example_name: str, parameters_path: Optional[str] = None) → Tuple[str, CompletedProcess]

pycomex.experiment.run_experiment(experiment_path: str, parameters_path: Optional[str] = None, blocking: bool = True, print_output: bool = True) → Tuple[str, CompletedProcess]

Given the string absolute experiment_path to the python experiment module, this function will execute that module as an experiment.

Parameters

• parameters_path (Optional[str]) – A string path to either a .JSON or a .PY file which specify parameter overwrite for the execution of the experiment.

• blocking (bool) – Whether to run the experiment in a blocking manner. If False, then the experiment will be started in the background.

• print_output (bool) – Whether to print the output of the subprocess to stdout

Returns

A tuple whose first element is the absolute string path ot the experiment archive folder which was created by the experiment. The second element is the subprocess.Process object which was used to execute the experiment.

pycomex.util module

Utility methods

class pycomex.util.CustomJsonEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

custom json encoder class which is used when encoding the experiment data into a persistent json file.

This specific class implements the serialization of numpy arrays for example which makes it possible to commit numpy arrays to the experiment storage without causing an exception.

default(value)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

class pycomex.util.Empty

Bases: object

class pycomex.util.RecordCode(stack_index: int = 2, initial_stack_index: int = 1, skip=False)

Bases: object

INDENT_SPACES = 4

This class can be used as a context manager to record code.

CHANGES 12.09.2022

Previously this class worked like this: In the __enter__ method a frameinfo supplied the line number at which the context starts and then the same was done in __exit__ and with Python 3.8 this actually worked The two methods returned the correct line numbers. But as of Python 3.10, this no longer works because the __exit__ method now ALSO returns just the line number of where the context manager starts.

But since we can still get the start line reliably, we just have to extract the code with some string processing now: With the starting line we know the indent of this context manager and can then record all the code which follows it in one level of indent deeper.

get_frame_info()

exception pycomex.util.SkipExecution

Bases: Exception

class pycomex.util.Skippable

Bases: object

pycomex.util.dict_value_sort(data: dict, key: Optional[str] = None, reverse: bool = False, k: Optional[int] = None)

pycomex.util.get_version()

pycomex.work module

class pycomex.work.AbstractWorkTracker(total_work: int)

Bases: object

property completed_work

estimate() → float

set_total_work(total_work: int)

start() → None

update(n: int = 1, weight: float = 1.0) → None

class pycomex.work.NaiveWorkTracker(total_work: int)

Bases: AbstractWorkTracker

estimate() → float

Module contents

Top-level package for pycomex.