Support generating LSIF and/or SCIP index.

[eownerdead]

Is your enhancement request related to a problem? Please describe.

Server Index Format (LSIF) and its successor SCIP Code Intelligence Protocol (SCIP) is a language-agnostic index format like ctags. It's developed by and used in sourcegraph, which provides web interfaces for code browsing with code navigation. Though there is already an LSIF implementation for Haskell, it has not maintained for long time.

Describe the solution you'd like

Adding a feature to generate SCIP index to this.
Also, we can get LSIF index with the SCIP to LSIF converter

Describe alternatives you've considered

Implement as a separated project.

Additional context

I know that rust-analyzer had this functionality, but I can't find any document...

Discussion at clangd

[fendor]

Hi, thank you for your feature request!

This looks like a nice thing to have!

[michaelpj]

Interesting. It seems to me that it should be possible to produce SCIP (which does seem better than LSIF, I never liked LSIF) from HIE files. That could be an independent library from HLS, which HLS could then use to simplify some of its internals. I think many cases where we use HIE files directly could instead use SCIP information, although I don't know if that buys us anything.

In some cases we could move functionality over, e.g. our new semantic tokens plugin could I think just be part of the SCIP indexer, since it contains very similar information. (cc @soulomoon )

I think this would be a nice standalone project. Probably too late to sneak it into GSOC? (There's even already a Haskell binding for SCIP!)

[soulomoon]

sounds interesting, I'll take a look.

---

Yes we can generate SCIP from HIE files.
And semantic tokens can depend solely on it.
But with one pitfall, hie file does not contain enough information to infer the Kind for a symbol if it is not defined in the file.

HLS use lookupKind to find this information.

If we solely base the generation on HIE files, I am not sure how much does it take us to generate hie files for all its dependencies(It need the hie file where the symbol is defined to get its Kind). And we might need to find a way to generate hie file for the base package ship with ghc as we have the same problem for the Goto dependency definition plugin #3749.

An alternative would be to add the Tything information (Not only the Type, since something is missing e.g. we can not tell if a name is pattern synonyms by looking at Type) to the HIE file, we use something like tyThingSemantic for the semantic token extraction) to capture some of the information we care about from TyThing and put the result into a new field of IdentifierDetails.

---

To prove the concept, we can make some changes like this to HieAst in ghc https://github.com/soulomoon/ghc/pull/1/files, then we can use just HieFile to create SCIP or semantic tokens

---

Alternatively we might expand the namespaces in ghc.
https://gitlab.haskell.org/ghc/ghc/-/issues/24544, chang to add EntityInfo to HieAst