Basic ExampleΒΆ
This is a basic example:
"""
This experiment will repeatedly create a text made of randomly sampled words.
The words are assembled into a text file, which is supposed to be saved as an
artifact of the computational experiment. Additionally, information such as the
total text length / run time of the calculations are to be saved as experiment
metadata.
This module-level doc string will automatically be saved as the description
for this experiment
"""
import os
import tempfile
import random
import textwrap
import urllib.request
from pycomex.experiment import Experiment
from pycomex.util import Skippable
# (1) All variables defined in uppercase are automatically detected as experiment
# variables and can be overwritten when externally executing the experiment
# using "run_experiment" for example
NUM_WORDS = 1000
REPETITIONS = 10
SHORT_DESCRIPTION = 'An example experiment, which shows all the basic features of the library'
with Skippable(), (e := Experiment(base_path=tempfile.gettempdir(),
namespace="example/basic", glob=globals())):
# (5) It is possible to assign an integer value to "work" which is (estimated)
# number of work packages to be completed as part of the experiment.
# This will be used to estimate the remaining time.
e.work = REPETITIONS
response = urllib.request.urlopen("https://www.mit.edu/~ecprice/wordlist.10000")
WORDS = response.read().decode("utf-8").splitlines()
# (1) The uppercase "experiment parameters" are stored in the "parameters"
# field of the experiment instance. Alternatively the variables can
# also just be used directly.
for i in range(e.parameters["REPETITIONS"]):
sampled_words = random.sample(WORDS, k=NUM_WORDS)
text = "\n".join(textwrap.wrap(" ".join(sampled_words), 80))
# (2) The first option to commit file artifacts to the experiment records
# is to use the "open" method directly to get a file manager context
file_name = f"{i:02d}_random.txt"
with e.open(file_name, mode="w") as file:
file.write(text)
# Alternatively there are convenience functions that accept various
# data types and handle the file creation automatically.
# e.commit_fig(file_name, fig) for pyplot figures for example
e.commit_raw(file_name, text)
# (3) Simple metadata (strings, numbers) such as various metrics can be
# stored to the internal experiment registry by simply indexing the
# experiment object. The slash '/' characters automatically define a
# nested structure.
# If a specific nested structure does not yet exist on assignment,
# it is automatically created first
text_length = len(text)
e[f"metrics/length/{i}"] = text_length
# >> e.data['metric']['length']['0'] = text_length
# (4) The "info" message should be used as an alternative to "print".
# These messages will be relayed to a Logger instance, which will
# print them to stdout, but also save them to a log file which is
# also stored as an experiment artifact.
e.info(f"saved text file with {text_length} characters")
# (5) Calling "update" signals the completion of one work package.
# This method will trigger a new estimation of the remaining time
# based on the time needed for previous work packages. This estimation
# is printed / logged.
e.update()
# The metadata is saved to an actual json file upon the content manager __exit__'s
if os.path.exists(e.path):
print(f"\n FILES IN EXPERIMENT FOLDER: {e.path}")
for path in sorted(os.listdir(e.path)):
print(os.path.basename(path))
When executed, the above file produces the following output:
$ python ../pycomex/examples/basic.py
2022-11-28 10:11:24,324 - ================================================================================
2022-11-28 10:11:24,324 - EXPERIMENT STARTED
2022-11-28 10:11:24,324 - namespace: example/basic
2022-11-28 10:11:24,324 - start time: Monday, 28 Nov 2022 at 10:11
2022-11-28 10:11:24,324 - archive path: /tmp/example/basic/000
2022-11-28 10:11:24,324 - debug mode? False
2022-11-28 10:11:24,324 - ================================================================================
2022-11-28 10:11:24,491 - saved text file with 7640 characters
2022-11-28 10:11:24,492 - (1/10) DONE - ETA: 2022-11-28 10:11:26.014703 (remaining time: 0.000h)
2022-11-28 10:11:24,494 - saved text file with 7613 characters
2022-11-28 10:11:24,494 - (2/10) DONE - ETA: 2022-11-28 10:11:25.183658 (remaining time: 0.000h)
2022-11-28 10:11:24,498 - saved text file with 7508 characters
2022-11-28 10:11:24,498 - (3/10) DONE - ETA: 2022-11-28 10:11:24.907173 (remaining time: 0.000h)
2022-11-28 10:11:24,501 - saved text file with 7582 characters
2022-11-28 10:11:24,501 - (4/10) DONE - ETA: 2022-11-28 10:11:24.768859 (remaining time: 0.000h)
2022-11-28 10:11:24,504 - saved text file with 7544 characters
2022-11-28 10:11:24,504 - (5/10) DONE - ETA: 2022-11-28 10:11:24.685502 (remaining time: 0.000h)
2022-11-28 10:11:24,506 - saved text file with 7461 characters
2022-11-28 10:11:24,507 - (6/10) DONE - ETA: 2022-11-28 10:11:24.629908 (remaining time: 0.000h)
2022-11-28 10:11:24,509 - saved text file with 7647 characters
2022-11-28 10:11:24,510 - (7/10) DONE - ETA: 2022-11-28 10:11:24.590275 (remaining time: 0.000h)
2022-11-28 10:11:24,512 - saved text file with 7720 characters
2022-11-28 10:11:24,512 - (8/10) DONE - ETA: 2022-11-28 10:11:24.560473 (remaining time: 0.000h)
2022-11-28 10:11:24,515 - saved text file with 7649 characters
2022-11-28 10:11:24,515 - (9/10) DONE - ETA: 2022-11-28 10:11:24.537285 (remaining time: 0.000h)
2022-11-28 10:11:24,518 - saved text file with 7646 characters
2022-11-28 10:11:24,518 - (10/10) DONE - ETA: 2022-11-28 10:11:24.518757 (remaining time: 0.000h)
2022-11-28 10:11:24,522 - ================================================================================
2022-11-28 10:11:24,522 - EXPERIMENT ENDED
2022-11-28 10:11:24,522 - start time: Monday, 28 Nov 2022 at 10:11
2022-11-28 10:11:24,522 - end time: Monday, 28 Nov 2022 at 10:11
2022-11-28 10:11:24,522 - duration: 0.000 hrs
2022-11-28 10:11:24,522 - error? None
2022-11-28 10:11:24,522 - ================================================================================
FILES IN EXPERIMENT FOLDER: /tmp/example/basic/000
00_random.txt
01_random.txt
02_random.txt
03_random.txt
04_random.txt
05_random.txt
06_random.txt
07_random.txt
08_random.txt
09_random.txt
analysis.py
annotations.rst
experiment_data.json
experiment_log.txt
experiment_meta.json
snapshot.py