Nodes performance

[jonesa99]

Hi, I'm starting to look into kwok and from a blog I read that "Kwok doesn’t accurately reflect the performance of real nodes because it approximates behaviors using simple formulas". Is it possible to find these formulas somewhere? And how can a fake node simulate a job? Do you need to also run some real nodes? Thanks for the help

[wzshiming]

Refer to https://kwok.sigs.k8s.io/docs/user/stages-configuration/

[jonesa99]

okay, and how can you estimate metrics like time if you don't know the data ut just the steps?

[wzshiming]

I think that can be from a label or field, or just a constant.

[jonesa99]

So I don't get what kwok is simulating. I can create different nodes and pods, assign a pipeline to them and get metrics like time and cpu. How are these metrics generated? And what are they useful for? In my mind it seemed to be useful to simulate the run of a pipeline even if my resources are not enough due to big data volume

[wzshiming]

I don't know how Argo collects this data, but there is always a way to simulate it, give Argo the data it needs, kwok doesn't care what it is.

https://kwok.sigs.k8s.io/docs/design/introduction/#user-stories

In fact, the main goal is unit testing and large-scale testing.

[wzshiming]

How are these metrics generated? And what are they useful for?

I can't answer this question at the moment, because I'm not sure how Argo works.

[jonesa99]

okay so in general it can be used to simulate cluster in large scale, but also in this case how could you simulate to have huge amount of nodes? how do the fake resources behave?

[jonesa99]

found it

[jonesa99]

sorry for not updating you, I made some progress and kwok seems useful for my case. I just have a quick question; in order to scale a pipeline using Argo I should expect the CPU and memory to increase when moving from 1 to 2 pods, while the execution time should decrease. Is it correct? And should I see this behavior also in kwok?

[wzshiming]

Yes, Simulation of CPU/Memory usage is expected to be supported in the next release.

[jonesa99]

okay thanks. so atm in kwok I can only see the difference in time execution when changing the number of pods, right?

[wzshiming]

I think the time you can get it directly from the Pod

[jonesa99]

yes I can do it, I am just wondering what are the possible metrics to collect

[wzshiming]

The metric also provides the execution time, but I don't know which one argo uses, the one from the Pod or the one from the metric.