Alternative implementation

[andsens]

Hi there. I needed a way to parse arguments in my bash scripts without external dependencies, so I wrote docopt.sh. It's an argument parser generator for docopt. I see it as an alternative to your current implementation and was wondering how I would go about getting the repo transferred to the docopt org?
I wrote an email to Vladimir, but he hasn't answered.

[CristianCantoro]

Hi there. I needed a way to parse arguments in my bash scripts without external dependencies, so I wrote docopt.sh. It's an argument parser generator for docopt. I see it as an alternative to your current implementation and was wondering how I would go about getting the repo transferred to the docopt org?
I wrote an email to Vladimir, but he hasn't answered.

Hi @andsens, from what I see you still need python for this implementation to work. So, it looks a lot like the previous implementation of docopts, see last python version in the releases. You can see the README.md of that version in the v0.6.1+fix branch. This version of docopts based on Go was born exactly out of the problem of not having to install any Python module dependiencies (see #9). Also, you may want to check out this other issue #7 to see some of the evolution of the current version of docopts.

How woud your version differ from that one?

[andsens]

docopt.sh itself depends on python, yes. But not the resulting script.
docopt.sh analyzes the docstring and generates a parsing tree for that specific docstring in pure bash and then inserts the resulting parser into the script.

Meaning that patched bash script is completely dependency free and works across all platforms.

EDIT: Basically, python is a compile-time dependency.

[Sylvain303]

Hi @andsens, I will look at your script and post back here when I know more about it. It could become a docopts companion, may be? I mean, may this repository can host your project side by side with docopts?

I suppose some code could be borrowed from one or the other.

docopts is supposed to be the portable bash version of docopt. The binary generated from golang, as awk sed and others bash binaries companion, can be copied on legacy server without any python support needed.

I also wrote a Proof of Concept docopts using JSON, but the parsing performance was bad.

Also, some nasty bug may never be fixed in python docopt lib anymore. So to achieve more goal I opted to use golang parser as a base. Which currently look like too much to the python one with the same nasty bug like this one #386

I will be more precise after reading your code.
Thanks for posting.

Regards,
Sylvain.

[Sylvain303]

So @andsens I tested your code this morning. It is fast and works well, great job. I far as I understand what you've developed, it seems you wrote your own parser to parse $DOC from bash source. And use python's docopt only for python argument parsing, right?

I like parsing too, but didn't have time yet to modify docopt parsing language yet. I feel that docopt should evolve to some point. I've found a couple of issue in docopt that require docopt parser to be modified like: better parse error handling #17 , also in golang docopt/docopt.go#61 , docopt/docopt#190 etc.

The original author doesn't seem to maintain the python's code anymore and the python parser seems to be the base for most parser. Including golang parser for docopt which do text object parsing internally exactly as the python's one, I feel it could been done by some more Go internal tricks, having object pointer to some parsed tree object or something that is not stored as string anymore.

Rereading docopt/docopt#371 I found that some recent docopt parser raised, it would be great if I could explore it too.

About, your request, what I propose you 2 things:

• Join docopts project as an alternate behavior. It becomes a project holding the two alternative, a python generator or a golang binary parser. Which could be nice, but also could disturb user. We could learn from both project and probably share our effort. We will need to adapt the project and each other.
• We ask around if someone else than Vladimir can create a project for you under docopt namespace. As I did to be granted as a maintainer of docopts which was finally granted after more than two years New maintainer needed #5. It could work and is less human challenging. ;-)

What's your feeling about that?

Regards,
Sylvain.

[CristianCantoro]

@andsens said:

docopt.sh itself depends on python, yes. But not the resulting script.
[cut]
EDIT: Basically, python is a compile-time dependency.

Oh, I see now. Thanks for the clarification. I had also time to try out docopt.sh now.

I thinkdocopt.sh is very interesting and complementary to docopts. As a user, I think both have their merits and different use cases. For one, I think I prefer how docopts works now because it is very clean and using it in a script is a minimal addition, but I can see the merit of having a pure-bash version.

I think it is worth to add docopts.sh to the docopt organization and both project should reference one-another.

I also think that the names are fitting because docopts is meant to be a replacement to GNU's getopts, while it makes sense that a pure-bash version of docopt is called docopt.sh.

[andsens]

Hey guys, thank you for the responses and thank you so much for trying it out! I agree that docopts and docopt.sh cover different use-cases:
docopts is a more clean approach and allows for easier modifiability, while docopt.sh would be needed for something like distributing other pure-bash based solutions (case in point: https://github.com/andsens/homeshick). Which is why I as well think that a separate repository where we cross-reference each other would make the most sense.

Regarding any bugs in the "reference implementation" @Sylvain303, I will try and see if I can reproduce the linked issue and then fix it. I agree that the developer side errors leave a lot to be desired, luckily for me that is something which can be fixed in python and for which I can send a PR back to docopt because the parser are pretty much the same.

[Sylvain303]

@andsens So for your repository creation, you may try to contact @mboersma who's is the person I contacted to be granted as maintainer for this repository in #5.

About parsing docopt language, so you confirm you've written you own parser for docopt.sh and not using docopt python's original to achieve your goal, right?

I'm interested in as I mentioned, where should be the best place to exchange about docopt parsing?

For the "Alternative implementation", I think we should provide a way to test our examples with both tools. I mean, a way to write example with a middle-ware function so the docopt parsing implementation could be swapable between the two. But the script behavior should stay unchanged.
So we could share example or bug via git repository and test.

What would you like as a method to increase compatibility for both project?
Should we create twin issues in both project to provide fix?

For example option name mangling.

Of course, this issue is probably not the place to discuss the example itself, but for a method of working together?

• option name mangle is different from docopts ≠ docopt.sh
  • docopts v0.6.1 python's code
  • I kept original docopts implementation for backward compatibility.

Also documented on both README docopts docopt.sh.

What would you suggest as method?

[andsens]

OK, I will contact @mboersma

About parsing docopt language, so you confirm you've written you own parser for docopt.sh and not using docopt python's original to achieve your goal, right?

No, I may have misspoken there. I started out with how python's docopt was implemented, removed unnecessary stuff for the python part and translated it for the bash part.

I'm interested in as I mentioned, where should be the best place to exchange about docopt parsing?

I would say wherever feels convenient/appropriate at the time, if we mention each others username it shouldn't make much of a difference.

For the "Alternative implementation", I think we should provide a way to test our examples with both tools. I mean, a way to write example with a middle-ware function so the docopt parsing implementation could be swapable between the two. But the script behavior should stay unchanged.
So we could share example or bug via git repository and test.

Now that would be super cool! I think we both have spent quite a bit of time on our respective tests. Something of particular note for my test: I test all bash versions that I say I support.

What would you like as a method to increase compatibility for both project?
Should we create twin issues in both project to provide fix?

Definitely. The problem may exist in both parsers, but the solution may be very different and the timeline for solving it as well. So twin issues makes the most sense.

option name mangle is different from docopts ≠ docopt.sh

The one for docopt.sh is pretty simple :-)
https://github.com/andsens/docopt.sh/blob/95d5d0d9adb525a5f30191ae52f85d2b08c7454b/docopt_sh/bash.py#L47-L48

Since nobody is relying on it yet I am free to change it up though.

Also: Regarding changing things in docopt.sh. One particular nice thing about the parser being embedded is that I don't need to worry about backwards compatibility so much. I even thought about getting it to check the version number when refreshing a parser and issuing warnings in case the developer needs to be made aware of any changes.

What would you suggest as method?

Regarding name mangling? I think predictability is key. Anything else, even usability, should be secondary. That's why I have chosen this rather crude method of just replacing unsupported characters with underscores. Anyone can very easily predict what the variable name is going to be and that makes using it so much easier.

[Sylvain303]

About parsing docopt language, so you confirm you've written you own parser for docopt.sh [...]

No, I may have misspoken there. I started out with how python's docopt was implemented, removed unnecessary stuff for the python part and translated it for the bash part.

Oh, interesting, so the parser in your code is mostly took from docopt.

I'm interested in as I mentioned, where should be the best place to exchange about docopt parsing?

I would say wherever feels convenient/appropriate at the time, if we mention each others username it shouldn't make much of a difference.

OK. I will probably create an issue on your project, for comparing behavior.

May be a wiki page about parsing would be a good place to explain docopt parsing language and referencing useful knowledge?

[...]

What would you suggest as method?

Regarding name mangling? I think predictability is key. Anything else, even usability, should be secondary. That's why I have chosen this rather crude method of just replacing unsupported characters with underscores. Anyone can very easily predict what the variable name is going to be and that makes using it so much easier.

No, my question was more general about method, for communicating about both project.
Starting with twin-issue, should be a good start, lets see how it gets.

I'm finishing #32 and I will come back to you about middle-ware for comparing behavior.

• Modify README.md to mention docopt.sh project (take care of name confusion)
• provide examples compatible for docopt.sh
• create twin-issue for name mangling compatible support

[Sylvain303]

Hi @andsens,

I created the twin issues, in order to test the method of working that way.
I'm also working on idea to provide compatible/convertible examples:

https://github.com/Sylvain303/docopt.sh/blob/master/examples/compatible.md

I don't know yet, if we need to support examples with :

• bash4 assoc array output
• docopts.sh auto parsing

We probably need to provide examples already written in compatible form at the base, instead of trying to write a converter handling most case.

I created a draft wiki page to work on the idea:

https://github.com/docopt/docopts/wiki/docopts---docopt.sh-comatible-examples

It may be more simple to summarize our ideas and goals from a wiki than in an issue discussion. Feel free to edit the wiki page.

[Sylvain303]

I crafted a workable example of compatible code.

Example prototype here.

The explanations details are updated in the wiki here.

Pre-built binaries here ==> v6.4.0-alpha1