Skip to content

Latest commit

 

History

History
135 lines (103 loc) · 7.19 KB

README-mlcommons.md

File metadata and controls

135 lines (103 loc) · 7.19 KB
Raw

StopWatch for MLCommons

Augmentation of codes for consideration into the inclusion of the science benchmarks, must use the

However, this library is ment for submisison to MLcommons and does not produce results that are easily readable.

We have used for a long time Clousmeh StopWatch that produces a nice table. Recently we have augmented it so that it can also create automatically mlcommons events. To leverage this feature we showcase here som simple examples of this integartion. The libarry is located at

To integarte is you can use

git clone https://github.com/mlperf/logging.git mlperf-logging
cd mlperf-logging
pip install -e .
pip install cloudmesh-common

A pytest documents its use

The elementary functions are

config = """
benchmark:
  name: Earthquake
  user: Gregor von Laszewski
  e-mail: laszewski@gmail.com
  organisation:  University of Virginia
  division: BII
  status: success
  platform: rivanna
  badkey: ignored
""".strip()

submitter = yaml.safe_load(config)

# activate the MLcommons logger
StopWatch.activate_mllog()

StopWatch.organization_mllog(**submitter)

# save an event with a value
StopWatch.event("test", values="1")

# save a dict in an event
data = {"a": 1}
StopWatch.event("stopwtch dict event", values=data)

# start a timer
StopWatch.start("stopwatch sleep")

# do some work
time.sleep(0.1)

# stop the timer
StopWatch.stop("stopwatch sleep")

# print the table
StopWatch.benchmark(sysinfo=False, tag="Earthquake", node="summit", user="Gregor", version="0.1")

The output will be

Name Status Time Sum Start tag msg Node User OS Version
test ok 0 0 2022-07-23 14:14:58 Earthquake summit Gregor Linux 0.1
stopwtch dict event ok 0 0 2022-07-23 14:14:58 Earthquake summit Gregor Linux 0.1
stopwatch sleep ok 0.1 0.1 2022-07-23 14:14:58 Earthquake summit Gregor Linux 0.1 
# csv,timer,status,time,sum,start,tag,msg,uname.node,user,uname.system,platform.version
# csv,test,ok,0.0,0.0,2022-07-23 14:14:58,Earthquake,None,summit,Gregor,Linux,0.1
# csv,stopwtch dict event,ok,0.0,0.0,2022-07-23 14:14:58,Earthquake,None,summit,Gregor,Linux,0.1
# csv,stopwatch sleep,ok,0.1,0.1,2022-07-23 14:14:58,Earthquake,None,summit,Gregor,Linux,0.1

To enable also a system information you can set syinfo to True and you will get a table similar to

Attribute Value
BUG_REPORT_URL "https://bugs.launchpad.net/ubuntu/"
DISTRIB_CODENAME focal
DISTRIB_DESCRIPTION "Ubuntu 20.04.4 LTS"
DISTRIB_ID Ubuntu
DISTRIB_RELEASE 20.04
HOME_URL "https://www.ubuntu.com/"
ID ubuntu
ID_LIKE debian
NAME "Ubuntu"
PRETTY_NAME "Ubuntu 20.04.4 LTS"
PRIVACY_POLICY_URL "https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
SUPPORT_URL "https://help.ubuntu.com/"
UBUNTU_CODENAME focal
VERSION "20.04.4 LTS (Focal Fossa)"
VERSION_CODENAME focal
VERSION_ID "20.04"
cpu AMD Ryzen 9 5950X 16-Core Processor
cpu_cores 16
cpu_count 32
cpu_threads 32
date 2022-07-23 14:14:58.264322
frequency scpufreq(current=2309.9574374999997, min=2200.0, max=3400.0)
mem.active 12.8 GiB
mem.available 111.1 GiB
mem.free 70.2 GiB
mem.inactive 37.9 GiB
mem.percent 11.6 %
mem.total 125.7 GiB
mem.used 12.8 GiB
platform.version #44~20.04.1-Ubuntu SMP Fri Jun 24 13:27:29 UTC 2022
python 3.10.5 (main, Jun 7 2022, 06:30:01) [GCC 9.4.0]
python.pip 22.1.2
python.version 3.10.5
sys.platform linux
uname.machine x86_64
uname.node summit
uname.processor x86_64
uname.release 5.15.0-41-generic
uname.system Linux
uname.version #44~20.04.1-Ubuntu SMP Fri Jun 24 13:27:29 UTC 2022
user Gregor

More advanced features include blocks that wrap start and sop methods so you do not have to call the separately

with StopWatchBlock("this is a block"):
    time.sleep(1.0)

with StopWatchBlock("this is a block with data", data=data):
    time.sleep(1.0)
    data["step"] = 1