Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fuseki] Reloadable configuration files #2462

Open
afs opened this issue May 10, 2024 · 5 comments · May be fixed by #2464
Open

[Fuseki] Reloadable configuration files #2462

afs opened this issue May 10, 2024 · 5 comments · May be fixed by #2464
Labels
enhancement Incrementally add new feature Fuseki

Comments

@afs
Copy link
Member

afs commented May 10, 2024

Version

5.0.0 dev

Feature

Add a server action that reloads the services defined in a configuration file and replace the server data services configuration with zero-downtime, no HTTP request lost or aborted while the server is running.

This allows:

  1. Reloading an in-memory graph with data from a file.
  2. Change the location of a TDB2 database
  3. Change of database configuration (e.g. (re)setting per dataset or per-endpoint query timeouts)

This is a subset of the configuration file capabilities. Some changes still require a server restart.

Limitation: transient data (in-memory, data loaded via HTTP) may be lost.

Are you interested in contributing a solution yourself?

Yes

@afs afs added enhancement Incrementally add new feature Fuseki labels May 10, 2024
afs added a commit to afs/jena that referenced this issue May 10, 2024
afs added a commit to afs/jena that referenced this issue May 12, 2024
afs added a commit to afs/jena that referenced this issue May 12, 2024
@afs afs linked a pull request May 12, 2024 that will close this issue
4 tasks
@flange-ipb
Copy link

Hello Andy,

this feature is very welcome, thanks for implementing!

Some ideas:

  • As an alternative to calling the reload endpoint, should it also be possible to send a signal to the Java process to achieve the configuration reload? That's typically done with SIGHUP.
  • Let's assume the configuration for text indexing has changed. It should be possible to rerun text indexing. That's probably out of scope of this issue, but maybe an idea for another endpoint?

@namedgraph
Copy link
Contributor

namedgraph commented May 12, 2024

Can't Fuseki simply detect when the config file changes? I think ontop is able to do that.
Calling an endpoint seems a bit like overkill here.

Looks like something called WatchService can detect file changes: https://docs.oracle.com/javase/tutorial/essential/io/notification.html

@flange-ipb
Copy link

WatchService could be unreliable, especially for non-local file systems. It seems there are even problems inside Docker containers. 😢

@rvesse
Copy link
Member

rvesse commented May 13, 2024

I've used Apache Commons IO Monitor APIs for watching the configuration file in the past. Though as @flange-ipb says it can be quite unreliable especially when running in a containerised environment

For example in a K8S environment where the config file is coming from a ConfigMap the way K8S actually mounts the file into the container means that the actual "file" ends up being a symbolic link and there's a hidden sub-directory lurking with the actual current version of the mapped file(s) from the ConfigMap bind mounted into your container

@rvesse
Copy link
Member

rvesse commented May 13, 2024

  • Let's assume the configuration for text indexing has changed. It should be possible to rerun text indexing. That's probably out of scope of this issue, but maybe an idea for another endpoint?

That functionality could be achieved by creating a custom FusekiModule though not sure if those are going to get called again when reloading a server or not, probably a question for @afs

afs added a commit to afs/jena that referenced this issue May 13, 2024
afs added a commit to afs/jena that referenced this issue May 17, 2024
afs added a commit to afs/jena that referenced this issue May 18, 2024
afs added a commit to afs/jena that referenced this issue May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Incrementally add new feature Fuseki
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants