Start Ydoc with the language server #9862

4e6 · 2024-05-03T16:32:03Z

Pull Request Description

related Investigate hosting yjs server within GraalJS #7954

Changelog:

update: Ydoc starts with the language server on the localhost:1234 by default. The hostname and ports can be configured by setting environment variables LANGUAGE_SERVER_YDOC_HOSTNAME and LANGUAGE_SERVER_YDOC_PORT
update: by default npm dev run uses the node Ydoc server. You can control it with POLYGLOT_YDOC_SERVER env variable. For example,
```
env POLYGLOT_YDOC_SERVER='true' npm --workspace=enso-gui2 run dev
```
To connect to the Ydoc server running on the 1234 port (the one started with the language server)
⠀
```
env POLYGLOT_YDOC_SERVER='ws://127.0.0.1:1235' npm --workspace=enso-gui2 run dev
```
To connect to the provided URL. Can be useful for debugging when you start a separate Ydoc process.
update: run npm install before the engine build. It is required to create the Ydoc JS bundle.

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

The documentation has been updated, if necessary.
All code follows the
Scala,
Java,
TypeScript,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
Unit tests have been written where possible.

build.sbt

Akirathan · 2024-05-13T08:29:46Z

engine/runtime-fat-jar/src/main/java/module-info.java

+  // ydoc-server
+  requires io.helidon.webclient;
+  requires io.helidon.webclient.websocket;
+  requires io.helidon.webserver;
+  requires io.helidon.webserver.websocket;


Good solution for now, if everything works with that. But at the end, we must ensure that runtime.jar does not require helidon modules. Let's keep that in mind and create a separate ticket for that before merging.

Akirathan · 2024-05-13T08:30:42Z

engine/runtime/src/main/java/org/enso/ClassLoaderConstants.java

@@ -13,9 +13,9 @@ public class ClassLoaderConstants {
   * consistent.
   */
  public static final List<String> CLASS_DELEGATION_PATTERNS =
-      List.of("org.graalvm", "java", "org.slf4j", "ch.qos");
+      List.of("org.graalvm", "java", "org.slf4j", "ch.qos", "io.helidon");


Good. This ensures that our boot class loader deleagates loading of all io.helidon.* classes to the system class loader, which picks the classes from JARs inside the component directory.

JaroslavTulach

It is great to see this to be green. Let's integrate and let's unblock the cloud integration.

somebody1234 · 2024-05-23T09:41:06Z

hey, just wondering - what's the roadmap for this stuff? will this be theoretically ready for integration with Cloud after this PR? or are there more steps that are needed?

4e6 · 2024-05-23T11:36:36Z

This should be enough to start integration in the cloud

4e6 · 2024-05-24T07:53:43Z

@Frizi waiting for your approval

PabloBuchu · 2024-05-24T08:09:35Z

| update: Ydoc starts with the language server on the localhost:1234 by default. The hostname and ports can be configured by setting environment variables LANGUAGE_SERVER_YDOC_HOSTNAME and LANGUAGE_SERVER_YDOC_PORT

Can you EXPOSE 1234 port in Dockerfile?

The port should be fixed there and both LANGUAGE_SERVER_YDOC_HOSTNAME LANGUAGE_SERVER_YDOC_PORT overridden (same as RPC_PORT and DATA_PORT)

4e6 · 2024-05-24T08:51:28Z

Sure, I'll update the Docerfile in the follow-up PR. This one has already too many stuff in it

AdRiley · 2024-05-28T09:54:33Z

lib/java/ydoc-server/src/main/java/org/enso/ydoc/Main.java

-      }
+    try (var ydoc = new Ydoc.Builder().build()) {
+      ydoc.start();
+      lock.acquire();


Is it intentional that this Semaphore is never released?

yes, we are waiting for any interruption signal here. Otherwise, the main process ends.

radeusgd · 2024-05-28T16:22:40Z

I realise that once this was merged, I can no longer compile the engine through sbt in IntelliJ, because I use the (recommended) version manager for Node: fnm. Unfortunately I was never able to set up IntelliJ to pick-up the environment changes that fnm generates to be able to 'see' node from an sbt shell in IntelliJ.

For now I just need to switch to using sbt from within PowerShell. This is okay, but a bit annoying as IntelliJ will start the sbt shell anyway to re-import the project from time to time.

Has anyone had a chance to set up IntelliJ to pick up the Node paths from fnm?

Akirathan · 2024-05-30T11:11:32Z

I realise that once this was merged, I can no longer compile the engine through sbt in IntelliJ, because I use the (recommended) version manager for Node: fnm. Unfortunately I was never able to set up IntelliJ to pick-up the environment changes that fnm generates to be able to 'see' node from an sbt shell in IntelliJ.

For now I just need to switch to using sbt from within PowerShell. This is okay, but a bit annoying as IntelliJ will start the sbt shell anyway to re-import the project from time to time.

Has anyone had a chance to set up IntelliJ to pick up the Node paths from fnm?

@radeusgd My IntelliJ's builtin sbt shell stopped working approx 9 months ago. I have switched to a separate sbt process in the terminal entirely. It is interesting that you have even made it so far.

- followup #9862 - related #7954 Expose the Ydoc port in the Docker container to make it configurable in the Cloud environment.

4e6 added the CI: No changelog needed Do not require a changelog entry for this PR. label May 3, 2024

4e6 self-assigned this May 3, 2024

4e6 force-pushed the wip/db/ydoc-language-server branch 2 times, most recently from 4acf4d6 to 22488da Compare May 10, 2024 13:28

Akirathan reviewed May 13, 2024

View reviewed changes

build.sbt Outdated Show resolved Hide resolved

Akirathan reviewed May 13, 2024

View reviewed changes

build.sbt Show resolved Hide resolved

Akirathan reviewed May 13, 2024

View reviewed changes

4e6 force-pushed the wip/db/ydoc-language-server branch from 5958a94 to d77d8a5 Compare May 13, 2024 15:59

4e6 added CI: Clean build required CI runners will be cleaned before and after this PR is built. and removed CI: Clean build required CI runners will be cleaned before and after this PR is built. labels May 13, 2024

4e6 force-pushed the wip/db/ydoc-language-server branch from d0f2108 to 8a16bf9 Compare May 17, 2024 06:16

4e6 and others added 15 commits May 17, 2024 07:16

feat: Ydoc.start

a0c39a8

feat: YdocTest

ab855fb

Exclude helidon modules from ydoc-server fat jar

d31c3c3

Run ydoc-server as a module

4896659

Fix some typos

4966272

Ability to run ydoc-server with ydoc-server/run.

a1b389d

Ydoc throws AssertionError if it cannot find the ydocServer.js resource

119c888

fmt

924006f

Add logback modules to ydoc-server module-path

9e1f954

Copy all helidon modules to the component directory

aafe692

Add chrome-inspector modules on ydoc-server module-path

3b83248

feat: build.sbt filter helidon jars

16289a7

feat: terrible hack

0074b67

feat: configure vite

eea99a0

fix: helidon version

8067e10

DEBUG: cleanup

c9f6562

JaroslavTulach self-requested a review May 21, 2024 13:32

fix: media type

bfd9e10

JaroslavTulach approved these changes May 21, 2024

View reviewed changes

4e6 added 2 commits May 23, 2024 07:43

Merge branch 'develop' into wip/db/ydoc-language-server

d6bc5d7

feat: verify licenses

4c6997a

somebody1234 requested a review from Frizi May 23, 2024 10:59

Frizi approved these changes May 27, 2024

View reviewed changes

4e6 added CI: Ready to merge This PR is eligible for automatic merge and removed CI: Ready to merge This PR is eligible for automatic merge labels May 27, 2024

JaroslavTulach mentioned this pull request May 27, 2024

Run ydoc-server with GraalVM #9528

Merged

5 tasks

Merge branch 'develop' into wip/db/ydoc-language-server

8bd081c

4e6 requested a review from marthasharkey as a code owner May 28, 2024 09:25

4e6 requested a review from radeusgd May 28, 2024 09:28

AdRiley reviewed May 28, 2024

View reviewed changes

AdRiley approved these changes May 28, 2024

View reviewed changes

revert: invalid change introduced in #9951

2820762

4e6 added the CI: Ready to merge This PR is eligible for automatic merge label May 28, 2024

mergify bot merged commit 858e646 into develop May 28, 2024
37 checks passed

mergify bot deleted the wip/db/ydoc-language-server branch May 28, 2024 13:51

hubertp mentioned this pull request May 29, 2024

Hotfix for finding parser library #10123

Merged

4e6 mentioned this pull request Jun 7, 2024

Expose Ydoc port in Docker container #10210

Merged

3 tasks

mergify bot pushed a commit that referenced this pull request Jun 7, 2024

Expose Ydoc port in Docker container (#10210)

2354532

- followup #9862 - related #7954 Expose the Ydoc port in the Docker container to make it configurable in the Cloud environment.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Start Ydoc with the language server #9862

Start Ydoc with the language server #9862

4e6 commented May 3, 2024 •

edited by somebody1234

Akirathan May 13, 2024

Akirathan May 13, 2024

JaroslavTulach left a comment

somebody1234 commented May 23, 2024

4e6 commented May 23, 2024

4e6 commented May 24, 2024

PabloBuchu commented May 24, 2024

4e6 commented May 24, 2024

AdRiley May 28, 2024

4e6 May 28, 2024

radeusgd commented May 28, 2024

Akirathan commented May 30, 2024

Start Ydoc with the language server #9862

Start Ydoc with the language server #9862

Conversation

4e6 commented May 3, 2024 • edited by somebody1234

Pull Request Description

Important Notes

Checklist

Akirathan May 13, 2024

Choose a reason for hiding this comment

Akirathan May 13, 2024

Choose a reason for hiding this comment

JaroslavTulach left a comment

Choose a reason for hiding this comment

somebody1234 commented May 23, 2024

4e6 commented May 23, 2024

4e6 commented May 24, 2024

PabloBuchu commented May 24, 2024

4e6 commented May 24, 2024

AdRiley May 28, 2024

Choose a reason for hiding this comment

4e6 May 28, 2024

Choose a reason for hiding this comment

radeusgd commented May 28, 2024

Akirathan commented May 30, 2024

4e6 commented May 3, 2024 •

edited by somebody1234