Avoid printing/parsing numbered args #4962

sellout · 2024-05-16T21:20:51Z

Overview

The Haskell data structures for various types are preserved through the “numbered args” processing, meaning that numbered args don’t need to be re-parsed after being expanded into the command. This avoids work, makes handling more consistent between commands, and allows for better error messages (say, when you refer to a branch when you need a definition).

Test coverage

This is a refactoring that should be largely covered by existing transcript tests that exercise the use of numbered args. All transcripts pass without modification.

As the type is changing to be more structured, we can’t use `[String]` in its place.

sellout

There is a bunch of cleanup to do in here (e.g., removing formatting done by wrong Ormolu version (but I think I have that sorted going forward)), but it’s feeling pretty solid with a few questions in the comments here.

I think there is some follow-up work in a separate PR to standardize some of the types expected by various commands, which would also reduce the handlers.

sellout · 2024-05-21T15:47:46Z

unison-cli/src/Unison/Codebase/Editor/HandleInput.hs

-    map (Text.unpack . Reference.toText . snd) types
-      <> map (Text.unpack . Reference.toText . Referent.toReference . snd) terms
+    map (SA.Type . snd) types
+      <> map (SA.Term . Referent.toReference . snd) terms


These cases (and other SA.Ref cases) had been displaying the Reference (hash), but we also have a HashQualified Name which seems like a much nicer thing to show to the user? Is there a reason that the HQ Name isn’t preferable here?

Aside: I looked in the diff for these lines (e.g. old line 1553), but it doesn't match what's shown here, and I don't understand why it wouldn't? I see it on old line 1302. I see Github's marked the snippet as "Outdated", but I don't think that would matter.

I think the answer to your question is that until #4907, the numbered args weren't shown to the user at all unless they ran debug.numberedArgs, making the question moot(?)

Aside: I looked in the diff for these lines (e.g. old line 1553), but it doesn't match what's shown here, and I don't understand why it wouldn't? I see it on old line 1302. I see Github's marked the snippet as "Outdated", but I don't think that would matter.

Ah, I think it’s because I squashed another commit into this one when it was in draft, but after I wrote this comment.

I had originally had Ref, Term, and Type as three separate StructuredArgument cases, but that just led to three copies of identical logic, so I removed the Term and Type cases until we can reasonably preserve that information.

I think the answer to your question is that until #4907, the numbered args weren't shown to the user at all unless they ran debug.numberedArgs, making the question moot(?)

Ah, right. I conflated a few things here and got confused.

But the question wasn’t really moot – if References are put into numbered args state, a command expecting a Name couldn’t then use a numbered arg. Whereas, if HashQualified Names were put into the state, the same command could then use it.

I don’t think it’s a change for this PR, but it seems like using the HQ instead of the Reference would help fix some of the existing numbered args problems.

Also, I think the pretty-printed numbered results lists (e.g., the output of find) could eventually lean more on the numbered args handling, rather that being constructed completely independently, which is another reason using HQ instead of a Reference would be a good change.

But I don’t know what the counter-arguments might be.

Yes, makes sense.

sellout · 2024-05-21T15:51:29Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

-helpFor p = I.help p
+helpFor = I.help
+
+handleProjectArg :: I.Argument -> Either Text ProjectName


These handlers aren’t great, IMO. There is so much code here. I feel like we can probably shrink the size of the StructuredArgument sum type more, which would help a bit here and there are some other possibilities, too, but we probably need to discuss.

sellout · 2024-05-21T15:53:48Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

+    ( \case
+        SA.Project project -> pure project
+        -- __FIXME__: Do we want to treat a project branch as a project?
+        SA.ProjectBranch (ProjectAndBranch (Just project) _) -> pure project


I doubt this was supported before, but when referencing numbered args, it seems like this kind of extraction is somewhat useful? E.g., you get back a list of all the ProjectAndBranches in your codebase, and then you run a command to rename a project, giving it some random ProjectAndBranch for that project – it makes sense that it would just ignore the branch name and rename the project … right?

Maybe? Sounds kind of weird to me, like if we said:

rename.project foo/main bar/branch

should that rename foo to bar?

I'd be more inclined to tell the user something like "hey I'm not sure we're on the same page here"

Yeah, @mitchellwrosen said the same. I think that’s right.

sellout · 2024-05-21T16:01:25Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

+    -- also: ShortHash.toText . Reference.toShortHash
+    Reference.toText termReference


All the formatting has been localized to this one function now, but some types had previously been formatted multiple ways, in which case I added a comment like this one.

All of the existing transcripts are working unmodified (although they definitely broke along the way), so at least as much as is covered is producing the same strings, but I’m not sure I always chose the best option.

sellout · 2024-05-21T16:03:42Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

+--
+--  __FIXME__: Don’t hardcode this
+schLength :: Int
+schLength = 10


This seemed to be the value used for short hashes in various places, but not exposed in a good way, AFAICT. A few of the handlers need this (for commands that expect short hashes where we have a full hash). But this feels like the wrong way.

Also, this might make sense for the formatter, but perhaps the commands that take ShortCausalHash or whatever should have it use the full length? (Even though the type has to be Short to allow users to type less.)

We were initially planning to derive this number from the individual codebase, like git does; but we never did.

I have a commit at the end now that handles this a bit differently. It avoids hardcoding, and preserves the full hash as an SCH in a number of cases. When the length is provided to the formatter¹ it takes advantage of it.

Footnotes

DumpNumberedArgs gets the hash length “from the DB”, and hopefully the command history could take advantage of it soon. ↩

This is the first step toward avoiding printing/parsing the values provided via `NumberedArgs`. It simply adds a new sum type to hold all of the types that can be in numbered args and stores it alongside the `Text` representation. It currently gets discarded when we actually expand the arguments.

This forces each `InputPattern.parse` function to serialize any `StructuredArgument` in its arguments. It’s a stop-gap that allows us to incrementally handle the structured arguments command-by-command.

This converts the commands to accept structured numbered arguments, rather than turning them all into strings.

Previously, the `Text` format had been preserved from the original code. This extracts all to a separate function that is called as needed. All transcripts still pass.

sellout · 2024-05-23T02:04:06Z

This still isn’t running any checks for me …

aryairani · 2024-05-23T03:42:14Z

This still isn’t running any checks for me …

At here I see

Workflows will not run on pull_request activity if the pull request has a merge conflict. The merge conflict must be resolved first.

I was unsure as to why this would matter, but reading a bit more I suppose it's because we run CI on pull_request (which it looks like uses the merge node, which can't exist with a conflict, to run the workflow), rather than on push, which just uses exactly whatever you committed. I didn't realize this! Or I forgot.

I could see a case for running both TBH, but we don't currently, probably due to running time.

It's too bad it doesn't even mention that it would have run if not for the merge conflict.

…d-args

mitchellwrosen · 2024-05-23T17:04:57Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

-helpFor p = I.help p
+helpFor = I.help
+
+handleProjectArg :: I.Argument -> Either Text ProjectName


Suggestion: use Either Pretty

aryairani

lgtm

unison-cli/src/Unison/CommandLine/InputPatterns.hs

aryairani · 2024-05-25T12:46:47Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

+    ( \case
+        SA.Project project -> pure project
+        -- __FIXME__: Do we want to treat a project branch as a project?
+        SA.ProjectBranch (ProjectAndBranch (Just project) _) -> pure project


Maybe? Sounds kind of weird to me, like if we said:

rename.project foo/main bar/branch

should that rename foo to bar?

I'd be more inclined to tell the user something like "hey I'm not sure we're on the same page here"

unison-cli/src/Unison/CommandLine/InputPatterns.hs

aryairani · 2024-05-25T13:38:46Z

unison-cli/src/Unison/CommandLine/InputPatterns.hs

+    $ \case
+      p : args ->
+        Input.FindI False . mkfscope
+          <$> first P.text (handlePathArg p)
+          -- __FIXME__: This changes things a bit. Previously, `find` and
+          --            friends would just expand the numbered args and search
+          --            for them like any other string, but now it recognizes
+          --            that you’re trying to look up something you already
+          --            have, and refuses to. Is that the right thing to do? We
+          --            _could_ still serialize in this case.
+          <*> traverse (unsupportedStructuredArgument "text") args


Could we see an example in a transcript for context?

If there's already a transcript that demonstrates find it could go there

I had already removed this change in other find cases, but overlooked this one. So I’ve changed this (and a couple other missed cases) to align with the others, which use unifyArgument to provide the string form of the structured argument.

# Conflicts: # unison-cli/src/Unison/Codebase/Editor/HandleInput.hs # unison-cli/src/Unison/CommandLine/InputPatterns.hs # unison-cli/src/Unison/CommandLine/OutputMessages.hs

aryairani · 2024-05-26T14:32:14Z

I want to understand the question about find, but apart from that it seems ready to merge — or did I miss addressing any other question?

This also generally improves formatting: - follows the longer line convention in Unison and - removes unnecessary `( … )` and `$` before `LambdaCase` args.

When `StructuredArgument`s are used as an input, preserve the entire hash. When printed, take the length as an optional argument (and show the full hash when unavailable).

aryairani · 2024-05-29T18:30:30Z

@sellout Go ahead and merge this when you're ready. (I was about to pull the trigger, but I had created a new merge conflict.)

…d-args

Use NumberedArgs type consistently

a41e8d0

As the type is changing to be more structured, we can’t use `[String]` in its place.

sellout force-pushed the structured-numbered-args branch from 918d3ea to 616744c Compare May 16, 2024 21:21

sellout commented May 21, 2024

View reviewed changes

sellout added 4 commits May 22, 2024 17:50

Push StructuredArguments into InputPatterns

8a95c5f

This forces each `InputPattern.parse` function to serialize any `StructuredArgument` in its arguments. It’s a stop-gap that allows us to incrementally handle the structured arguments command-by-command.

Have InputPatterns handle NumberedRef

0a94308

This converts the commands to accept structured numbered arguments, rather than turning them all into strings.

Serialize StructuredArguments on demand

d6a394f

Previously, the `Text` format had been preserved from the original code. This extracts all to a separate function that is called as needed. All transcripts still pass.

sellout force-pushed the structured-numbered-args branch from ced8f2b to d6a394f Compare May 23, 2024 01:40

sellout marked this pull request as ready for review May 23, 2024 02:03

sellout requested a review from mitchellwrosen May 23, 2024 02:04

sellout added 2 commits May 22, 2024 23:12

Merge remote-tracking branch 'upstream/trunk' into structured-numbere…

9d5ee42

…d-args

Allow structured args in find commands

ff785cb

sellout mentioned this pull request May 23, 2024

Type the main arg to execute #5003

Merged

mitchellwrosen reviewed May 23, 2024

View reviewed changes

This was referenced May 25, 2024

docs 1 on result of find doesn't work #3939

Closed

Docs name #5012

Merged

aryairani approved these changes May 25, 2024

View reviewed changes

Merge branch 'trunk' into structured-numbered-args

1667c59

# Conflicts: # unison-cli/src/Unison/Codebase/Editor/HandleInput.hs # unison-cli/src/Unison/CommandLine/InputPatterns.hs # unison-cli/src/Unison/CommandLine/OutputMessages.hs

sellout added 4 commits May 28, 2024 14:34

Allow structured args in more find commands

647072d

Don’t allow ProjectBranch as project arg

c6f1f2c

Have handlers fail with Pretty, not Text

a9c29d0

This also generally improves formatting: - follows the longer line convention in Unison and - removes unnecessary `( … )` and `$` before `LambdaCase` args.

Handle SCH carefully in StructuredArguments

f8474ff

When `StructuredArgument`s are used as an input, preserve the entire hash. When printed, take the length as an optional argument (and show the full hash when unavailable).

sellout added the ready-to-merge Apply this to a PR and it will get merged automatically once CI passes and 1 reviewer has approved label May 28, 2024

sellout mentioned this pull request May 29, 2024

delete.term with numbered arg fails and with a cryptic message #4898

Closed

Merge remote-tracking branch 'upstream/trunk' into structured-numbere…

e9d2a21

…d-args

sellout merged commit 6805135 into unisonweb:trunk May 29, 2024
19 checks passed

sellout deleted the structured-numbered-args branch May 29, 2024 23:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid printing/parsing numbered args #4962

Avoid printing/parsing numbered args #4962

sellout commented May 16, 2024 •

edited

sellout left a comment

sellout May 21, 2024 •

edited

aryairani May 26, 2024 •

edited

sellout May 28, 2024

aryairani May 28, 2024

sellout May 21, 2024

sellout May 21, 2024

aryairani May 25, 2024

sellout May 25, 2024

sellout May 28, 2024

sellout May 21, 2024

sellout May 21, 2024

aryairani May 23, 2024 •

edited

sellout May 28, 2024

sellout commented May 23, 2024

aryairani commented May 23, 2024 •

edited

mitchellwrosen May 23, 2024

sellout May 28, 2024

aryairani left a comment

aryairani May 25, 2024

aryairani May 25, 2024

aryairani May 25, 2024

sellout May 28, 2024

aryairani commented May 26, 2024

aryairani commented May 29, 2024

		-- also: ShortHash.toText . Reference.toShortHash
		Reference.toText termReference

Avoid printing/parsing numbered args #4962

Avoid printing/parsing numbered args #4962

Conversation

sellout commented May 16, 2024 • edited

Overview

Test coverage

sellout left a comment

Choose a reason for hiding this comment

sellout May 21, 2024 • edited

Choose a reason for hiding this comment

aryairani May 26, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aryairani May 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Footnotes

sellout commented May 23, 2024

aryairani commented May 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aryairani left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aryairani commented May 26, 2024

aryairani commented May 29, 2024

sellout commented May 16, 2024 •

edited

sellout May 21, 2024 •

edited

aryairani May 26, 2024 •

edited

aryairani May 23, 2024 •

edited

aryairani commented May 23, 2024 •

edited