-
-
Notifications
You must be signed in to change notification settings - Fork 102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce expansion from type and contract generation. #633
base: master
Are you sure you want to change the base?
Conversation
8436289
to
048db6e
Compare
Whoa |
@@ -43,3 +43,8 @@ | |||
;; | |||
;; Also, this type works better with inference. | |||
(-> (make-Prompt-Tagof Univ (-> Univ ManyUniv))))))) | |||
|
|||
(begin-for-syntax |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment w/ brief description?
@@ -78,6 +78,11 @@ | |||
|
|||
(define-syntax (-#%module-begin stx) | |||
(syntax-parse stx | |||
[(mb e0 e ...) | |||
#:when (eq? '#:no-add-mod (syntax-e #'e0)) | |||
#'(#%plain-module-begin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment?
(dynamic-require (module-path-index-join '(submod "." #%type-decl) m) | ||
#f)))) | ||
|
||
(provide add-mod! do-requires) | ||
(define (adjust p) | ||
(match p |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment? is this just sexp -> sexp? what is it adjusting and why?
[_ p])) | ||
|
||
(define (->mp mpi submod) | ||
(collapse-module-path-index (module-path-index-join `(submod "." ,submod) mpi))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
signature?
@@ -399,8 +407,13 @@ | |||
(if (from-typed? typed-side) | |||
(and/sc sc any-wrap/sc) | |||
sc)) | |||
;(eprintf "predef: ~s ~s ~s\n" predef-contracts type typed-side) | |||
(cached-match |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
printfs -- fyi
@@ -996,6 +1010,9 @@ | |||
(define extflnonnegative? (lambda (x) (extfl>= x 0.0t0))) | |||
(define extflnonpositive? (lambda (x) (extfl<= x 0.0t0)))) | |||
|
|||
(require (submod "../static-contracts/instantiate.rkt" predefined-contracts)) | |||
;(hash-set! predef-contracts (cons Univ 'typed) (cons #'any/c 'flat)) | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is this commented out hash-set!
?
(define (function-contract? stx) | ||
(syntax-case stx () | ||
[(arr . _) | ||
(and (identifier? #'arr) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is really low importance, but I've been trying to avoid using arr
because people so easily confuse "arrow", "arity", etc and conversations about them can be really confusing if taken literally when they are used incorrectly. maybe it doesn't matter here.
@@ -479,6 +479,7 @@ | |||
(define/with-syntax (new-defs ...) defs) | |||
(define/with-syntax (new-export-defs ...) export-defs) | |||
(define/with-syntax (new-provs ...) provs) | |||
(do-contract-requires) | |||
(values |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is it important that do-contract-requires
appears here (and not earlier or later)? Does it just need to come before the get-contract-requires
below? If so, would it be clearer to have a function that just does both in the right order? (i.e. does the work of do-contract-requires
and then returns the result from get-contract-requires
?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or if not, maybe a comment about why this effectful procedure is called there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, do-contract-requires
is for side-effect before any contracts are generated, so it has to be earlier. I'll find the logical place to put it an add a comment. get-contract-requires
doesn't depend on those side effects, and just needs to be called to put the output in the correct submodule.
(-Arrow doms rng)) | ||
|
||
(define (simple->values doms rng) | ||
(->* doms (make-Values (map -result rng)))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I read this identifier as "simple to values"... but its really more like "simple arrow with values" or something, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. This just exists so that the expansion of type serialization is smaller.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks great -- I made a few comments about adding some prose and/or signatures to help future maintainers.
One question/comment -- I was surprised to see how much RAW looking code is now appearing in typed-racket-more
(e.g. typed-racket-more/typed/racket/draw.rkt
). I worry that we're exposing more implementation details outside of TR's implementation, and there should be some simpler API that we instead define and maintain. Are we really expecting maintainers of other libraries to write stuff like that in their adapter modules?
@samth can you say a little bit about high-level strategy here? I'm of the naive opinion that when one writes a module in typed racket that exports, say, identifiers f and g with types T S, then there could be another module made at that same point that contains, roughly, I'm clearly missing something (possibly not so) subtle and I'm definitely not on the critical path, but I am curious how wrong my understanding is, so if you have the time to fill me in, that'd be great. |
@rfindler What you describe as your naive opinion is in fact the case currently, although it's a bit more complicated than that. Roughly, a module like this:
currently expands to the following (with a few simplifications):
Note that I've left out how the One question you might have at this point is why all the submodules. First, the Another question you might have is what does So, where does duplication come in? Well, imagine that you had two modules, both of which have an That's the background, and in the next comment I'll write about how this PR changes things to reduce duplication. |
I see I was not clear in my writing. From what I can tell, the compiled version of this program:
contains an entire copy of the (I will note that this is not like the duplication you discuss; I didn't write out the Frame% type in this file!) |
(PS: it also seems to contain copies of many other contracts; I think I see the text% contract and the dc<%> contract in there ... any maybe more?) |
Ok, now that you've read the background, here's the high-level strategy:
Oh, and we're going to do the same thing for types as well, since the serialization of types can be large. With that said, here's the outline of the new expansion (note that this isn't fully implemented yet):
Several things have changed here. First, we've added definitions of The new mutations set up two new tables, which are mappings that tell us that if we want an expression that evaluates to the type Given that, what can we do now? Consider this module:
Now, when we go to generate a contract for However, there's one more issue. Lots of Typed Racket files don't depend on any other modules written in We've already faced this with type serialization, and the solution is to initialize the table with some commonly-used types so there's no duplication at all. That's what's going on in the manually-constructed |
This doesn’t seem to take advantage of the fact that some of the types
already have names and thus the contracts can also have corresponding names
(without having to make guesses about when to make a C).
Is that not needed? Not helpful?
Does this strategy always avoid duplicating a contract for any type that is
already just a name?
|
Right, this doesn't use yet another table, which implements the meaning of Also, there's another wrinkle, which is that for classes, we have type names for the class types, but most of the contracts refer to the instances, and sadly those don't share a contract. Note that the part where it generates the contents of the |
Just a point of clarification: when you write "....and sadly those don't share a contract", you mean that there isn't a type name? Surely if one had a contract that corresponds to the type Frame%, then one could just write Regardless, that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(This is in-progress
right?)
- Is
#:no-add-mod
being used because#lang typed/racket
doesn't includetyped/racket/gui/base
etc.? - Are many other modules going to need to change like the ones here in
typed-racket-more
? If they do need changes, would it be possible to just changetyped-racket/base-env/extra-env-lang
to those "user" files can look prettier?
[(app (lambda (t) (hash-ref predef-contracts (cons t typed-side) #f)) | ||
(? values con-id)) | ||
;(eprintf "found a match ~s ~s\n" con-id type) | ||
(impersonator/sc (syntax-local-introduce con-id))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of always making an impersonator/sc
, will this eventually check the kind and make a flat or chaperone when possible?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes.
Yes, if other modules use |
@rfindler Unfortunately, it really can't always reuse class contracts to produce instance contracts. There are a few reasons for this. One is that when an instance is provided from Typed Racket, we need to use an opaque object contract; that's what happens in this program:
A second reason is that there's not just one contract for a given type, there's both the contract when the value comes from untyped code and the one when the value goes to untyped code. (There's also a third one when the same contract has to handle both.) Fortunately, this isn't really a source of much duplication, because most modules have instances in their interfaces, rather than classes, and because the individual method contracts can be shared. |
I've improved some of the bigger problems with this PR, and it's become clear that implementing the remaining big piece will be harder than I thought, so I'm considering moving forward with just this part first, especially since it's already a big win. I still plan to address the outstanding comments, of course. |
That sounds good to me. I'm interested to see the build plot (http://build-plot.racket-lang.org/) after this change, but I haven't gotten around to adding a way to make a build plot using your "reduce-expansion" branch instead of the one that pkgs.racket-lang.org reports. |
I've fixed the use of |
It looks at a glance like library sizes (at least
|
The changes in
Here's a spreadsheet with details from a particular directory in |
Two files from Here's a diff from Here's a diff from |
These diffs indicate two problems:
|
Ok, another problem. Starting DrRacket with this change causes the following error (some of the module language code is in Typed Racket). What does this error indicate, and what am I doing wrong?
|
The serialization in |
I'll have to investigate the |
Commit racket/racket@ab7dffa fixes the |
* Mark all multiply-referenced types as popular, but avoid extraneous popular types cause by them being contained in other popular types. Thanks to @mflatt for help with this. * Create some more simple constructors to use in generated code. * Define some popular types as pre-defined. Together, this reduces the size of framework-types.rkt's zo file by about 100k.
This also sets up the infrastructure for sharing contracts between modules that use TR, but doesn't use that infrastructure automatically yet, just for explicitly predefined types. Reduce .zo size for plot-gui-lib by about a factor of 11.
Since GUI types are rarely used in practice on typed/untyped boundaries (they were mostly removed from plot) it's not clear that adding all of them to the zo size of "typed-racket-more" is the right thing. This makes the current change just set up the infrastructure.
After those changes (thanks @mflatt) |
Unsafe undefined is now exported but has a comment about how it’s unclear
whether or not it’s useful to export. The comment should just go away at
this point probably, right?
…On Tue, May 22, 2018 at 4:49 PM Sam Tobin-Hochstadt < ***@***.***> wrote:
After those changes (thanks @mflatt <https://github.com/mflatt>)
lazy-snip-typed_rkt.zo is now 5.4k. Next step is to fix the excessive
requires.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#633 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADfg6qgGig2xDwR1T_TrxUaCLJjT47Qsks5t1HnugaJpZM4P8UEl>
.
|
I probably won't add that commit in this branch, but it was useful to have
at the moment.
On Tue, May 22, 2018 at 5:06 PM, Andrew Kent <notifications@github.com>
wrote:
… Unsafe undefined is now exported but has a comment about how it’s unclear
whether or not it’s useful to export. The comment should just go away at
this point probably, right?
On Tue, May 22, 2018 at 4:49 PM Sam Tobin-Hochstadt <
***@***.***> wrote:
> After those changes (thanks @mflatt <https://github.com/mflatt>)
> lazy-snip-typed_rkt.zo is now 5.4k. Next step is to fix the excessive
> requires.
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <#633 (comment)
>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ADfg6qgGig2xDwR1T_
TrxUaCLJjT47Qsks5t1HnugaJpZM4P8UEl>
> .
>
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#633 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAO786R_bXJUgVBJSHV68mQlHpAnfO_Rks5t1H3HgaJpZM4P8UEl>
.
|
This reduces zo size in
plot-gui-lib
by about 11x.