You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I haven't tested it super thoroughly, but I saw Lambda runs till max timeout even thought it couldn't process anything. One example was sending gz files which I realized not supported
Steps to reproduce (if applicable)
Steps to reproduce the behavior:
deploy indexer lambda
send compressed file, test.json.gz
Expected behavior
It should exit immediately and not run until timeout
Configuration:
export class RustFunction extends Function {
constructor(scope: Construct, id: string, props?: Partial<FunctionProps>) {
const lambdaAssetPath = path.join(
__dirname,
"../../quickwit/quickwit",
"quickwit-lambda/deploy",
id
);
console.log("lambdaAssetPath", lambdaAssetPath);
super(scope, id+'_f', {
...props,
code: Code.fromAsset(lambdaAssetPath),
handler: id, // use id to specify either "indexer" or "searcher"
runtime: Runtime.PROVIDED_AL2,
architecture: Architecture.ARM_64,
logRetention: RetentionDays.ONE_DAY,
tracing: Tracing.DISABLED,
});
}
}
Binaries compiled with this command (I didn't use cross it was too slow on Mac)
It would be a useful improvement indeed, even though it is far from trivial to identify all un-recoverable errors and bubble them up to stop the lambda.
One example was sending gz files which I realized not supported
gz files are supported, if you run Quickwit Lambda with these packages it should work just fine.
In general, we don't guaranty that main or the Quickwit release tags (e.g 0.8.1) builds to a functioning AWS Lambda. The release cycle of Quickwit Lambda is still independent at this stage. You should use the Lambda release tags instead (aws-lambda-beta-xx).
Agree, won't be easy.
I saw it runs server-side software. Pretty remarkable you made it work in Lambda, and so well.
Is there an option to make all errors unrecoverable?
If users prefer Lambda can use own retry mechanism.
I recognize, though - this is probably not how quickwit servers designed :)
Is there an option to make all errors unrecoverable?
There is currently a retry mechanism at the indexing pipeline level that could be disabled. That wouldn't be a silver bullet solution but might already avoid hanging in many failure cases.
Describe the bug
I haven't tested it super thoroughly, but I saw Lambda runs till max timeout even thought it couldn't process anything. One example was sending gz files which I realized not supported
Steps to reproduce (if applicable)
Steps to reproduce the behavior:
Expected behavior
It should exit immediately and not run until timeout
Configuration:
Binaries compiled with this command (I didn't use cross it was too slow on Mac)
LIBZ_SYS_STATIC=1 TARGET_CC=aarch64-linux-musl-gcc RUSTFLAGS="-C linker=aarch64-linux-musl-gcc -C link-arg=-static -C opt-level=z -C lto" cargo build --release --target aarch64-unknown-linux-musl
quickwit --version
checked out v0.8.1
The text was updated successfully, but these errors were encountered: