Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running code in gophernotes is much slower than using go run #250

Open
123quant opened this issue Jun 16, 2022 · 4 comments
Open

Running code in gophernotes is much slower than using go run #250

123quant opened this issue Jun 16, 2022 · 4 comments

Comments

@123quant
Copy link

Is there any solution

@mccolljr
Copy link

go run compiles your code to a native executable and runs that, but Gophernotes is using an interpreter under the hood (gomacro, a link is in the README for this package). There is always going to be some amount of slowdown when interpreting a language vs. running a native executable.

Having said that - It may be helpful to share some specific code, since there could possibly be ways to change it to be more performant in the gophernotes environment

@cosmos72
Copy link
Member

cosmos72 commented Jun 16, 2022

Go toolchain and standard library do not support (and actually makes really hard to) JIT-compile code at runtime and execute it.
I have tried that road, and desisted for several reasons:

  • the garbage collector crashes if it finds a non-Go function running on the Go stack. You'd need to run JIT-compiled functions on the C stack (and switching stacks is slow).
  • the garbage collector only knows about memory allocated by Go runtime (= Go memory), and will not scan for pointers the memory allocated by malloc() or other means (= C memory etc.).
  • the Go specs say you can store pointers to Go memory only in Go memory, and you can use them only from Go: you cannot store them in C memory, and you cannot use them in JIT-compiled code.

For these reasons, almost all Go interpreters use reflection to manipulate Go data structures, and the interpreted code is executed either as closures, as a bytecode interpreter, or (at worst) AST tree walking.

All these techniques have an overhead, and from some of my (old) benchmarks gophernotes and gomacro are among the fastest Go interpreters available, averaging to 10-100 times slower than compiled Go.

As @mccolljr pointed out, the speed depends a lot from the code you run:
in general, code dealing only with builtin types (bool, int*, uint*, float*, string) is quite fast, while using composite types (arrays, chans, maps, pointers, slices, structs) is relatively slow.

Note: there is an interpreter gijit that compiles Go source code to Lua. It's fast but has to to a lot of convolutions to manipulate Go data structures.

@cosmos72
Copy link
Member

cosmos72 commented Jun 16, 2022

Update: imported packages are automatically compiled by gophernotes/gomacro using Go toolchain, and loaded as plugins (this works on Linux and Mac OS X only - unluckily on Windows the procedure is more convoluted).

So if there's a particular piece of code that your really need to be fast, you can for example:

import "github.com/my/package/full/path"

or even import it from a local filesystem path (useful in case the package is not published):

import "/my/package/absolute/path"
import "./my/package/relative/path"
import "../my/package/relative/path"

then use your favorite package from gophernotes, and it will be compiled to native code.

@123quant
Copy link
Author

thanks for answer

go run将您的代码编译为本机可执行文件并运行它,但 Gophernotes 在后台使用解释器(gomacro,此包的自述文件中有一个链接)。在解释语言与运行本机可执行文件时,总会有一些减速。

话虽如此 - 共享一些特定代码可能会有所帮助,因为可能有办法将其更改为在 gophernotes 环境中更高性能

thanks for answer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants