You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A periodic slow down is observed when the InferRequest.infer are continually called, when the OpenVINO inference engine is single threaded and bounded to CPU 0 with all-ones input. The peaks in the inference times are evidently away from the average, and regularly spaced as shown in the following figure.
The phenomenon is reproducible for a much slower model in figure, with almost the same peak positions along the wall-clock time with different starting point.
The inference engine is expected to behave without large peaks or fluctuations.
Step-by-step reproduction
"""model from openvinotoolkit/open_model_zoo/tree/master/models/inteltaskset -c 0 python3 main.pyPython 3.8.13numpy==1.24.2openvino==2023.3.0Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHzCentOS Linux release 7.9.2009 (Core)3.10.0-1160.102.1.el7.x86_64"""importtimeimportopenvino.runtimeasov# 2023.3importnumpyasnpMODEL_PATH='person-attributes-recognition-crossroad-0230.xml'NUM_THREADS=1PRECISION=ov.Type.f32DEVICE_NAME='CPU'BATCH_SIZE=1WARMUP=30# in unit of secondsBENCHMARK=600# in unit of secondsdefcreate_workload_fn(model_path: str):
config=dict()
config.setdefault(ov.properties.inference_num_threads(), NUM_THREADS)
config.setdefault(ov.properties.hint.inference_precision(), PRECISION)
config.setdefault(
ov.properties.affinity(), ov.properties.Affinity.NONE)
core=ov.Core()
model=core.read_model(model_path)
compiled_model=core.compile_model(
model=model, device_name=DEVICE_NAME, config=config)
infer_request=compiled_model.create_infer_request()
inputs=list()
forinput_layerincompiled_model.inputs:
shape=input_layer.partial_shape.get_min_shape()
shape[0] =BATCH_SIZEdtype=input_layer.element_type.to_dtype()
value=np.ones(shape=shape, dtype=dtype)
inputs.append(value)
defworkload_fn():
_=infer_request.infer(inputs=inputs)
returnworkload_fndefmain():
workload_fn=create_workload_fn(MODEL_PATH)
expiry=time.perf_counter_ns() +int(WARMUP*1e9)
whileexpiry>time.perf_counter_ns():
workload_fn()
timestamps=list()
start=time.perf_counter_ns()
expiry=start+int(BENCHMARK*1e9)
current=starttimestamps.append(current)
whileexpiry>current:
workload_fn()
current=time.perf_counter_ns()
timestamps.append(current)
x_data=list(map(lambdax: (x-start) *1e-9, timestamps[1:]))
y_data=list(map(
lambdax, y: (x-y) *1e-6, timestamps[1:], timestamps[:-1]))
# # plot from x_data and y_data# import matplotlib.pyplot as plt# fig, ax = plt.subplots()# ax.plot(x_data, y_data, '.-')# ax.set_xlabel('wall-clock time / s')# ax.set_ylabel('inference time / ms')# ax.grid()# fig.savefig('main.png', dpi=300)if__name__=='__main__':
main()
Issue submission checklist
I'm reporting a performance issue. It's not a question.
I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
There is reproducer code and related data files such as images, videos, models, etc.
The text was updated successfully, but these errors were encountered:
OpenVINO Version
2023.3.0
Operating System
Other (Please specify in description)
Device used for inference
CPU
OpenVINO installation
PyPi
Programming Language
Python
Hardware Architecture
x86 (64 bits)
Model used
models/intel/person-attributes-recognition-crossroad-0230 in open_model_zoo
Model quantization
No
Target Platform
lscpu
lscpu -e
Performance issue description
A periodic slow down is observed when the
InferRequest.infer
are continually called, when the OpenVINO inference engine is single threaded and bounded to CPU 0 with all-ones input. The peaks in the inference times are evidently away from the average, and regularly spaced as shown in the following figure.The phenomenon is reproducible for a much slower model in figure, with almost the same peak positions along the wall-clock time with different starting point.
The inference engine is expected to behave without large peaks or fluctuations.
Step-by-step reproduction
Issue submission checklist
The text was updated successfully, but these errors were encountered: