Proof of concept: Custom functions

[jphastings]

I got the software equivalent of an earworm and decided to turn it into a proof of concept.

👀 I'm seeking feedback of any kind, which could include "this isn't something Hugo is interested in"; I'll be happy to turn this sorry state of affairs into quality, tested code, or to close this line of thinking down.

What is it?

Code similar to this would allow Hugo users to write and use custom Typescript (or Javascript) functions for use inside their templates.

{{ $today := (now.Format "Mon January 2" ) }}

{{ printf $today }} → Mon June 12            # Standard Hugo
{{ fn "enc.Rot13" $today ) }} → Zba Whar 12  # New!

See the external.md at the top of the PR for more detail.

Can I test it out?

Yes!

1. Build this branch
2. Clone my demo site
3. Edit layouts/index.html and functions/*.ts
4. Run ../path/to/built/hugo in the site root
5. Check out public/index.html to see the output

Why is this useful?

A lot of Hugo's Github Issues seem to be about changing or adding new functions to Hugo. Though contributing to Hugo's codebase is an admirable, it's a long-term solution to these requests and likely isn't available option to many.

Writing and sharing function files, storing them in the site files (eg. in functions/*.ts), is a cross-platform, simple, accessible way to be able to write and share helper functions.

Some important benefits:

• The JS runtime used is totally sandboxed (no access to the filesystem, web or similar), so malicious custom code can't do harm.
  • I'd like to be able to use a limited fetch, but there are lots of new challenges there.
• Unlike using something like Go plugin files, this TS/JS is platform agnostic (so it can be committed to source code along side the site files)
• Other JS modules should be import-able*, so custom code writers can build on the shoulders of the JS community. (*Caveat: I haven't tried this, it'd likely be a little complex right now)
• JS execution happens entirely in Go (no v8 here!) and is time-limited. It'll be slower than Go, but not "slow".

Why this PR isn't at all ready

I've thrown this together to understand if it's interesting to the community. There are lots of sharp edges. Some of the ones I've found/considered but not addressed:

• The error reporting if you've written bad TS/JS is plain bad or absent.
  • Good dev tooling here will be crucial, or making these function files will be very hard
• Go types are auto-cast to Javascript ones, which mostly works (with strings, numbers, structs, even times—though timezone data is lost) but there are undoubtedly arguments people could pass to a JS function that'd be super confusing to interact with.
• The function call is a little clunky ({{ fn "filename.FunctionName" "args" }} instead of {{ filename.FunctionName "args" }}), as Hugo seems to require namespaced functions to be methods in Go and, because these are dynamically generated from JS, I can't do that (without substantial codechange)
  • This also means that the code I wrote to extract exported constants of the name <FunctionName>Examples, expecting it to be in the [][2]string format used for other function examples, is useless right now.
• While Goja (the Javascript-VM-in-Go library used here) seems stable as an open source project, go-typescript seems less well maintained.
• I'm re-compiling the JS every time it's called. I can definitely optimise this.

Thanks for reading this far! Let me know what you think.

[CLAassistant]

All committers have signed the CLA.

[bep]

You may want to have a look at https://github.com/bep/gojap

Creating a new runtime for each function is bound to be very slow, memory hungry.

I like the idea of this (given the above, I have been thinking about it ...), but there are lots of caveats, and I suspect a proof of concept would have a better chance of success if it tested out the hard parts first, mainly:

• In templates, It's very rare that I have simple problems (addition, date formatting ...) that I think "oh, I wish I could do this in JavaScript".
• So for this to be useful, I suspect many people would expect to be able to pass the Hugo Page, Site etc. down to the JS runtime. That brings up several issues, but the main part is to get it working.

[jphastings]

Thank you for the rapid reply @bep! For me this was indeed hard thing #1 ("is it at all possible"), but you're absolutely right, passing .Site or similar large objects over is definitely a hard thing worth testing! I can now build some of the trivial things I'd like to have (for example: custom calendar date formatting), but nothing more complex.

The two things I'll work on next:

1. Making use of gojap (it looks great!) to reduce the number of runtimes (perhaps one per file, or one total — I'll try to measure resource usage)
2. Getting a demo of what'd be needed for passing something more substantial over. Do you have an example use-case in mind? (I was considering doing something like implementing a MakeRSS function — would this cover the challenges you see?)

That challenge I faced of all function calls needing to be Go methods; do you think this is worth addressing in a proof of concept? I'm not particularly familiar with the Hugo codebase, so I'd be very willing to leave it to a later round of thought (if we get there).

[bep]

That challenge I faced of all function calls needing to be Go methods; do you think this is worth addressing in a proof of concept?

You don't want to tackle all problems -- but what I would have wanted to see is something like (pseudo code, I just made up these, so they're not particulary clever):

function getLastMod(pages) {
   // return the newest .Date from the page list given
}

function filter(pages, predicate) {
  // return the pages matching the predicate func.
}