How to modify max seq len in android llama demo app?

[CHNtentes]

Hi,
I tried android llama demo app. While it works, I don't know how to make generated text longer.
I already set max length to 4096 when exporting pte model, and when I use llama_main cli I can set seq_len to 4096.
How do I modify this value in demo app code?

[afwcole]

Hey I ran into this issue aswell. You have to edit the sequence length in the jni_layer_llama.cpp file. can be found in the following folder "extension/android/jni/jni_layer_llama.cpp" for the generate function.

It is set to 128 but you will have to change it 4096
jint generate( facebook::jni::alias_ref<jstring> prompt, facebook::jni::alias_ref<ExecuTorchLlamaCallbackJni> callback) { runner_->generate( prompt->toStdString(), **4096,** [callback](std::string result) { callback->onResult(result); }, [callback](const Runner::Stats& result) { callback->onStats(result); }); return 0; }

Here is the link to the line you have to change here