Questions about future plans

[samhocevar]

Hi, I plan to work on a few features / changes and I’d like to know your opinion about which ones would have a chance to land in your project, and which ones I should keep in my fork because they go beyond the scope you have in mind.

• syntax support: as you may have seen, ImGuiColorTextEdit gives more control to the user, for instance with regexes to define their own syntax colouring; are you planning to go this route, or do you wish to keep this feature minimal?
• tooltips: I would like to let my users hover on a function name (maybe recognised by the syntax driver) and see a popup with its documentation or other notes
• option to compile without the file watcher mechanism (it pulls quite a lot of system-specific dependencies, as well as the experimental libstdc++fs) or without file support altogether (the host application should be able to decide whether loading/saving is allowed, and what to do with the editor buffer)
• option to compile on platforms without threads (right now the ThreadPool constructor just throws on these platforms)
• option to compile without SDL (editor_imgui.cpp still uses it for some scancodes macros)
• real tab support (or at least an option to not replace them with spaces)
• make up/down arrows move according to the physical cursor location rather than the column number (causes weird effects when using a variable-width font)

[cmaughan]

Hi,
I'll try to answer your points :) Zep is a part time code and coffee project for me, and has been designed to support a live coding project that I'd like to do, so I have ongoing plans to add whatever I need to make it a useful IDE editor. That said, I only get a bit of time to work on it - Christmas was a good time for me to get things in a pretty useful state!. I suggest filing issues for each thing you'd like to see; I'll start doing the same so there is a good list of work items. To answer your points:

• Syntax Support. I have some reservations about adding regex, since as the ImGuiColorTextEdit author notes, this is slow; and IMHO difficult to use (some of those regex expressions are huge). That said, I can see it's more flexible. So far I haven't had much need to use anything other than keyword recognition - and the bracket matching that the rainbow parentheses code suggests; I wonder if ImGuiColorTextEdit does anything more than that in the end? But extending/improving the syntax coloring is not a problem if necessary; it would be fairly trivial to write a new highlighter that used regex instead as an option; or replaced the current one.
• Tooltips. That's something I want to add. You'll note that I have some initial rudimentary 'Marks' which can be tested with :ZTestMarkers. I had hoped to add a side car window, like a Vim QuickFix list as one way to do this; enabling quick switching through compile issues, etc. But agreed that tooltips would be useful. I don't see a problem here; its just an extension that adds a tooltip to a marker range; and some code to draw the tip box over it. I suspect it is only 30 mins work or so.
• The Filewatcher should be cross platform; it compiles different backends depending on OS. But sure, it is only used to spot when the user hand edits the config file and auto-update the editor. I don't see a problem with adding a #define to remove it (IIRC, I already to that on Mac)
• The syntax highlighter used to use threads, but I turned it off because it wasn't really a bottleneck, and it had a bug. Again, no issues with adding a #define to disable threads; I don't believe they are used anywhere at the moment.
• It's an oversight that SDL uses some scancodes; that needs fixing.
• Real tabs: I don't know how hard that is! It probably isn't too bad.
• Variable width fonts: I'm curious to see this in action; that's another one where I'm not sure how hard it is to do; there are a few places I assume fixed width. But it can be fixed I'm sure.

I actually tried to build your repo this morning before work, but fell over a bit trying to get the submodules installed. On a fresh repo, a git submodule update fails on the 'lolengine' bit; I tried it on Linux and Windows. I also tried manually installing the repo, but had other issues with windows SDK versions, etc. I will figure it out later and have a look!
I've played with Pico8 before, and am a fan; so looking forward to playing with your IDE :)

[samhocevar]

Thanks for the detailed answer! About the zepto8 repo, you probably want git submodule update --init --recursive because the lol submodule itself has submodules. Speaking of that, is there a reason why ImGui isn't a submodule in Zep?

[samhocevar]

Also, you can test an HTML build of zepto8 with Zep support (compiled with emscripten). The editor isn’t linked with the emulator yet, so changing the code won’t do anything, but all the Zep features should be here (the only thing that has an effect right now is using the RAM editor at addresses 0x6000 — 0x8000 to change pixel values).

[cmaughan]

Regarding building it; I figured out it is because I don't have any SSH keys on my machine. It's been a long time since I've used SSH to connect to git; I always clone with HTTPS. But your submodule requires SSH and that's why it fails. That's another reason I don't use submodules I guess :) To answer your question, I use the subtree merge strategy for git. Before I did that, I used to always be fighting with submodules. Here's a good article which I often refer to when I can't remember the commands!

https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree

I tried your HTML build; impressive :) I will have to play with emscripten.

[samhocevar]

Ah, thanks for investigating that! It was an error on my side, the submodule URI was using the SSH protocol explicitly rather than a relative path… It’s now fixed.

[cmaughan]

The last thing seems to be that there is nothing in src\z8lua:
Severity Code Description Project File Line Suppression State
Error C1083 Cannot open source file: 'z8lua/lzio.c': No such file or directory libz8lua D:\Dev\zepto8\src\c1xx 1
Error C1083 Cannot open source file: 'z8lua/lvm.c': No such file or directory libz8lua D:\Dev\zepto8\src\c1xx 1
Error C1083 Cannot open source file: 'z8lua/lundump.c': No such file or directory libz8lua D:\Dev\zepto8\src\c1xx 1

[samhocevar]

Ah, sorry, I made the above fix in a hurry and I think I broke several submodules… also, I hadn’t tested the Visual Studio build and some files were missing. It’s all fixed now.

[cmaughan]

e7687d4 Fixes the Thread one I believe?

[samhocevar]

e7687d4 Fixes the Thread one I believe?

Yes! I’ll strike it out.

[cmaughan]

#21 covers tooltips :)

[cmaughan]

I looked at the FileWatcher, and I agree that it probably doesn't work outside win/mac/linux. But I don't think it uses "filesystem". You may have that confused with what zep does:

• includes and uses "filesystem" on platforms that support it (non apple currently!)
• uses its own bare-bones implementation of "filesystem", in mfilesystem.h

It's probably possible to set the necessary define in file.h and correctly fallback to mfilesystem on your platform?

That said, I would like to get to the point where file load/save and directory queries are done via a user-supplied interface or a default. That way clients of zep could supply their own fake file system to map to their database, etc. or use a standard one that talks to the native file system.

[samhocevar]

e7687d4 Fixes the Thread one I believe?

Actually, the threading issue is still not fixed. I would argue it’s a bug in ThreadPool.hpp:

ThreadPool(size_t threads_n = std::thread::hardware_concurrency()) : stop(false)
{
    if(!threads_n)
        throw std::invalid_argument("more than zero threads expected");

I think threads_n should be std::thread::hardware_concurrency() - 1.

Right now if threads_n == 1 it means the platform only supports the main thread, so the thread pool should not attempt to spawn an additional one.

[cmaughan]

Yeah, I’m actually working on removing it - it isn’t necessary at all.

[cmaughan]

Yeah, I’m actually working on removing it - it isn’t necessary at all.

I realized that the threadpool was useful for the syntax highlighter because it saves the thread create cost every time you modify the text. So I've hopefully fixed it to make it immediately run anything that is added on a system that doesn't have threads; making the rest of the code look the same. I check hardware concurrency for 0 or 1 in that case. Otherwise, it gets scheduled.

[cmaughan]

Zep now has an abstract file system, and the file watcher is not part of the code; clients have to pass the changes to the IFileSystem when necessary.

[cmaughan]

Next on my list for Zep is cleaner/more robust markers for highlights, etc. I want to integrate the markers with the syntax highlighting, so they work alongside the theming. This is mainly because I have another unpublished project for a live shader editing tool, and I need better error marks, etc.
I also want to add some kind of support for remembering the dead spaces in the editor pane, and/or the ability to insert 'editor lines' that have nothing to do with the text. This is for adding live widgets/markers to the text.

[cmaughan]

Real tabs are now supported.