Feature: Statically checked declaration-site variance #524
Comments
eernstg
added
the
feature
|
First draft of feature specification at PR dart-lang/sdk#557. |
|
The "exact" type already exists in Dart in certain places - the type of a literal or object creation expression may have an exact type, which is why For use-site invariance, we will have both As stated, the type arguments can only be used in type pattern expressions (type annotations), not class expressions or literals. The following should be allowed: List<exact String> foo(List<exact int> list) => (list..add(42)).map((x) => "$x").toList();It is meaningful and useful. What about: class C<T> {
List<exact T> mkList() => <T>[];
}Is this meaningful? Maybe. If you have a List<T> mkList() => <T>[];will never have a static return type of So, we need some substitution rules for type arguments to make this work. It also suggests that a lot of platform libraries should be changed to have That suggests a migration issue. If we change some platform libraries to return, say, If we need migration anyway, then we don't need to make the syntax backwards compatible, and we could do something else, like: class C<+T , -S, =U> { // out T, in S, inout U
List<+T> mkList() => <T>[]; // not exact T
List<U> mkList() => <U>[]; // exact U
}We do get the inherent complexity of a two-tier type system where it matters whether you write I expect For declarations site variance, we already have something similar for function types. For those, the variance is automatically computable from the co-/contra-variance of the occurrences of the type arguments. |
Right. The notion of an exact type is not specified, but it is used in some cases by our tools (even to raise an error for certain "impossible" casts). The notion of a type argument which is known exactly is a different concept (an object can have dynamic type
Both of those are instantiations of the generic type class A<X, Y> {}
class B<X> implements A<X, int> {}With that, We may or may not want to let
I'd prefer to say that a type accepts type arguments, each of which may have the modifier
class C<T> {
List<exactly T> mkList() => <T>[];
}That can be allowed, and would be safe, but the computation of the member access type must take into account that the statically known value of main() {
C<num> c = C<int>();
List<num> xs = c.mkList(); // Static type of `c.mkList()` is `List<num>`.
List<exactly num> ys = c.mkList(); // Error (downcast).
}Of course, with
I believe the most straightforward way to get this right is to consider the types of member accesses, with a case for each variance of the type parameters of the class. Type parameters with no variance modifier are the most complex ones, of course, because they are allowed to occur in any position in a member signature.
Indeed.
Right, but
I think that wouldn't ideally be such a common situation: The declaration site variance modifiers are required to match the usage (so if you have a type parameter The use of I would expect the reasonable trade-off to be somewhere between just using the legacy types as they are (with the current level of type safety, which has been used in practice for years without complete panic) and teasing out the very last drop of static type safety, by adding |
eernstg
changed the title
|
I more like the +T and -T syntax in Scala |
|
The current |
The primary beneficiary is the USER. The void addToList<T>(List<T> target, List<T> src) {
for(var x in src) {
target.add(x);
}
}
void test(List<num> nums) {
addToList(nums, [3, 4, 5]);
}
void main() {
List<double> doubles = [3.5];
test(doubles);
} |
Reading between the lines of this question, I think you are missing the primary effect of this feature, which is that it changes the subtyping relation between types. Describing |
I think tatumizer is suggesting |
Ah, sorry, I misunderstood your question. Here's the intution:
Does that help?
Neither. Classes are allowed to use type variables everywhere in a method, which corresponds most closely to
For completeness, here is the example rewritten to use an example of an invariant implementation of import 'dart:collection';
class Array<inout T> extends ListBase {
List<T> _store;
int get length => _store.length;
void set length(int i) => _store.length = i;
T operator[](int index) => _store[index];
void operator[]=(int index, T value) => _store[index] = value;
Array.fromList(List<T> this._store);
}
void addToList<T>(Array<T> target, Array<T> src) {
for(var x in src) {
target.add(x);
}
}
void test(Array<num> nums) {
Array<num> ints = Array.fromList([3, 4, 5]);
addToList(nums, ints);
}
void main() {
Array<double> doubles = Array.fromList([3.5]);
test(doubles);
} |
|
@tatumizer I think this is getting a bit far afield - variance is definitely a confusing subject, but I don't think we're going to fix that here. Is it fair to summarize your general take here as being that you find |
|
repeat the |
|
@leafpetersen I think the |
mit-mit
moved this from Being implemented
to Ready for implementation
in Language funnel
|
That's a very good example because However, we do have some covariant occurrences of the relevant type variables as well, so we'd need to make some classes invariant in order to get the statically safe typing, or we can keep them unchanged (using dynamically checked covariance) because they are already "mostly OK", and they are difficult to change (like Here is an example (using some fake classes just to make the example small): // Will probably continue to use dynamically checked covariance.
// Could also be invariant, but that's a massively breaking change.
class Stream<T> {
Future pipe(StreamConsumer<T> _) => Future<Null>.value();
}
// Can be soundly contravariant (if `Stream` is dyn-covariant).
class StreamConsumer<in T> {
Future addStream(Stream<T> stream) => Future<Null>.value();
}
// Needs to be invariant (to get a safe `add` and a safe `stream`).
class StreamController<inout T> extends StreamConsumer<T> {
Stream<T> get stream => Stream<T>();
void add(T t) {}
}
void main() {
final StreamController<num> numbers = StreamController();
final StreamController<double> doubles = StreamController();
doubles.stream.pipe(numbers); // Sure, no problem! ;-)
doubles.add(123);
}With these changes, As usual, we can emulate invariance (which is not quite as flexible as the real declaration-site variance feature where we can use contravariance where that's sufficient and invariance only where nothing else will do). Here's a minimal example, where we only declare those few methods that we're actually calling here: import 'dart:async' as async;
// Provide invariant types for some stream related classes.
typedef Inv<X> = X Function(X);
typedef Stream<X> = _Stream<X, Inv<X>>;
typedef StreamConsumer<X> = _StreamConsumer<X, Inv<X>>;
typedef StreamController<X> = _StreamController<X, Inv<X>>;
extension type _Stream<T, Invariance extends Inv<T>>(async.Stream<T> it) {
Future pipe(StreamConsumer<T> consumer) => it.pipe(consumer.it);
Stream<R> cast<R>() => Stream(it.cast<R>());
}
extension type _StreamConsumer<T, Invariance extends Inv<T>>(
async.StreamConsumer<T> it
) {
Future addStream(Stream<T> stream) => it.addStream(stream.it);
}
extension type _StreamController<T, Invariance extends Inv<T>>._(
async.StreamController<T> it
) implements _StreamConsumer<T, Invariance> {
_StreamController() : this._(async.StreamController<T>());
Stream<T> get stream => Stream(it.stream);
void add(T t) => it.add(t);
}
// Usage, in some other library (assume `import 'something';)
void main() {
final StreamController<num> numbers = StreamController();
final StreamController<double> doubles = StreamController();
// doubles.stream.pipe(numbers); // Compile-time error.
doubles.stream.cast<num>().pipe(numbers); // OK.
doubles.add(123);
}Note that Finally, we should be able to use extension types together with statically checked declaration-site variance in order to be able to access the // Provide soundly variant types for some stream related classes.
extension type StreamConsumer<in T>(async.StreamConsumer<T> it)
implements async.StreamConsumer<T>;
extension type StreamController<inout T>._(
async.StreamController<T> it
) implements async.StreamController<T> {
StreamController() : this._(async.StreamController<T>());
}[Edit, Sep 19, 2023: Add the bound |
|
That's very cool, thanks so much for sharing! Are any of the same tricks possible for replicating contravariance and static covariance in current Dart? |
|
I've been playing around with the experimental feature and it's awesome for these situations. I do, however, keep forgetting what FWIW I would cast my vote for modifiers (as mentioned above) which make this explicit:
I have also stumbled into some situations where an |
|
@dnys1 wrote:
The invariance emulation won't work out of the box: Contravariance should allow subtyping relationships like typedef _Ctv<X> = void Function(X); // `_Ctv<num> <: _Ctv<int>`.
typedef MyConsumer<X> = _MyConsumer<X, _Ctv<X>>;
class _MyConsumer<X, Contravariance> {
void m(X x) {}
}
void main() {
MyConsumer<int> intConsumer = MyConsumer<num>(); // Error, should have been OK.
}We can't express statically checked covariance, either, because it's just going to be dynamically checked covariance as usual, and there will not be any errors for things like It's possible that there is some other technique, but someone would have to invent it, and tells us about it. ;-)
Right, that's always a difficult point. With respect to the subtype relationships, I tend to think that
We've considered using
Sure, the covariant/contravariant/invariant positions in a type can occur in a return type, because that return type can have enough structure on its own to have both covariant and contravariant positions: class C<in X> {
void Function(X) get myGetter => (X x) {...};
}You could say that In general, the variance of a position is defined recursively (PDF, see section 'Variance'), so you can always stack enough function types on top of each other to obtain a covariant/contravariant position anywhere in a function signature. So it's not really a grey area, it's just a situation where the "inversion" that occurs whenever we switch from a function type to one of its parameter types occurs in a type that occurs in a member declaration signature, rather than at the top level of the signature. We can also invert twice and get back to covariance: class D<out X> {
final X x;
D(this.x);
X Function() myMethod(void Function(X) arg) => () => x;
} |
|
Regarding your proposed workaround, the following case won't work: typedef Invariance<T> = T Function (T);
typedef Foo<T> = _Foo<T, Invariance<T>>;
final class _Foo<T, I> {
const _Foo(this.bar);
final _Bar<T> bar;
void baz() => bar.baz(this);
}
final class _Bar<T> {
void baz(Foo<T> foo) => print(foo);
}I get the following error: I could fix it by changing -final class _Foo<T, I> {
+final class _Foo<T, I extends Invariance<T>> {The error disappeared and I still get a static error if I try to pass covariant values to Do you think this change is valid or can it introduce some other unexpected behavior? |
|
@mateusfccp, you are right. I actually used that approach a while ago (March 2023: dart-lang/sdk#51680 (comment)), I just got lazy in the meantime because the actual examples did not require this extra bit of type information. ;-) But we can do it: It is benign for clients to have the bound, because they should always be using actual type arguments of the form It would still be possible to violate that invariant inside the declaring library, which means that it isn't precisely equivalent to the real language mechanism, but the emulation is better when we include this bound. I changed the comment where the idiom is introduced accordingly. Thanks! |
|
I am sorry if I am being annoying with this workaround, but I got to another problem while using it in my project. Consider the following: typedef Invariance<T> = T Function (T);
typedef Foo<T> = _Foo<T, Invariance<T>>;
abstract interface class _Foo<T, I extends Invariance<T>> {
void baz();
}
mixin DefaultFoo<T> on Foo<T> {
@override
void baz() {}
}
final class Bar with DefaultFoo<Object> implements Foo<Object> {}This code shows me two compile-time errors:
I managed to fix the first error by also redirecting the mixin: +typedef DefaultFoo<T> = _DefaultFoo<T, Invariance<T>>;
+
-mixin DefaultFoo<T> on Foo<T> {
+mixin _DefaultFoo<T, I extends Invariance<T>> on _Foo<T, I> {However, the second error persists, and I couldn't find a way to circumvent this... I feel like Is there a way or is it impossible to use this workaround with mixins? |
No problem, of course! Try this: typedef Invariance<T> = T Function (T);
typedef Foo<T> = _Foo<T, Invariance<T>>;
abstract interface class _Foo<T, I extends Invariance<T>> {
void baz();
}
typedef DefaultFoo<T> = _DefaultFoo<T, Invariance<T>>;
mixin _DefaultFoo<T, I extends Invariance<T>> on _Foo<T, I> {
@override
void baz() {}
}
final class Bar extends Foo<Object> with DefaultFoo<Object> {}Perhaps you didn't actually mean that the mixin should be typedef Invariance<T> = T Function (T);
typedef Foo<T> = _Foo<T, Invariance<T>>;
abstract interface class _Foo<T, I extends Invariance<T>> {
void baz();
}
typedef DefaultFoo<T> = _DefaultFoo<T, Invariance<T>>;
mixin _DefaultFoo<T, I extends Invariance<T>> implements _Foo<T, I> {
@override
void baz() {}
}
final class Bar with DefaultFoo<Object> implements Foo<Object> {}It's worth noting that we must use In that case you'll have to admit to the world that the phantom type parameter ( However, that just illustrates that we do need declaration site variance. ;-) |
|
@leafpetersen wrote:
In order to demonstrate this point: Micro-benchmark which measures the cost of covariant parameter type checks in Dart VM/AOT.import 'package:benchmark_harness/benchmark_harness.dart';
const int N = 1000000;
class A {}
class B extends A {}
class C1<T> {
@pragma('vm:never-inline')
void foo(List<T> x) {
}
}
class C2<T> {
@pragma('vm:never-inline')
void foo(List<T> x) {
}
}
class TypeCheckBench extends BenchmarkBase {
final C1<A> obj = int.parse('1') == 1 ? C1<B>() : C1<A>();
final List<A> arg = int.parse('1') == 1 ? <B>[] : <A>[];
TypeCheckBench() : super('TypeCheck');
@override
void run() {
for (int i = 0; i < N; ++i) {
obj.foo(arg);
}
}
}
class NoTypeCheckBench extends BenchmarkBase {
final C2<A> obj = C2<A>();
final List<A> arg = <A>[];
NoTypeCheckBench() : super('NoTypeCheck');
@override
void run() {
for (int i = 0; i < N; ++i) {
obj.foo(arg);
}
}
}
void main() {
final benchmarks = [TypeCheckBench(), NoTypeCheckBench()];
for (var benchmark in benchmarks) {
benchmark.report();
}
}In this benchmark, an empty instance method with one In one case ( Results: JIT, x64: AOT, x64: JIT, arm64: AOT, arm64: With statically checked declaration-site variance, if type parameter |
|
Any updates on the status of this feature for release? |
|
I don't think the team has any ETA, but the feature is behind an experimental flag ( It doesn't seems to be something the team is prioritizing, tho. |
and others
This is the tracking issue for introducing a statically checked mechanism for declaration-site variance in Dart.
Background: The original request motivating this kind of feature is dart-lang/sdk#213; the initial proposal for declaration-site invariance is dart-lang/sdk#213. The initial proposal for the related feature known as use-site invariance is dart-lang/sdk#229, and the corresponding tracking issue is dart-lang/sdk#753.
Note that this issue does come up in practice. Here are a few examples gathered since April 2023:
not a subtype ofissue sdk#52826Object-inference induced type errors. #3156(int) => voidis not a subtype of type(dynamic) => voidsdk#53523The text below describes properties of this feature which are good candidates for being adopted. Many things can still change, and a full feature specification will be written and used to manage the discussions about the final design.
Variance in Dart Today
As of Dart 2.4 or earlier, every type variable declared for a generic class is considered covariant. The core meaning of this is that a parameterized type
C<T2>is a subtype ofC<T1>wheneverT2is a subtype ofT1. Other subtype rules can then be used to show subtype relationships likeList<int> <: Iterable<dynamic>andMap<String, String> <: Map<Object, Object> <: dynamic.This type rule is not sound; that is, in order to maintain heap soundness it is necessary to check certain types dynamically. This means that a program with no compile-time errors can fail with a type error at run time.
For instance, with the declaration
List<num> xsand some expressionewith static typenum, it is necessary to check during evaluation ofxs.add(e)that the value ofeactually has the type which is required byxs: It is possible that it is aList<int>or even aList<Never>, and it would then be a dynamic type error if the value ofeis adouble, even though the expression had no type errors at compile-time.Dynamically checked covariance enables many software designs that would be rejected by a traditional statically checked approach to variance (e.g., as in Java or C#). This allows developers to make a trade-off between more flexible types (e.g., a variable of type
List<num>is allowed to refer to aList<int>) in return for accepting the potential dynamic type errors (aList<int>will work safely under the typeList<num>in a lot of ways, just not all).We want to enable a statically checked typing discipline for variance as well (rejecting more programs, but providing a compile-time guarantee against the run-time type errors described above). This feature is concerned with the provision of support for that.
Declaration-site Variance
Declaration-site variance can be used to declare a strict and statically checked treatment of variance for each type variable of a generic class.
Syntactically, declaration-site variance consists in allowing each type parameter declaration of a generic class declaration to include one of the following modifiers:
out,in, orinout. We say that such a type parameter has explicit variance.The use of type parameters with explicit variance in the body of the enclosing class is restricted. It is a compile-time error for a type variable marked
outto occur in a non-covariant position in the signature of a member declaration; and for a type variable markedinto occur in a non-contravariant position. For example:Here are some core properties of declaration-site variance:
We obtain the following subtype relationships: Let
Cbe a generic class with one type parameterX. Assume thatSis a subtype ofT. IfXis markedoutthenC<S> <: C<T>; ifXis markedinthenC<T> <: C<S>. Note that there is no subtype relationship betweenC<S>andC<T>ifXis markedinout, unlessS == T.A type parameter with explicit variance can be used in the specification of a superinterface. For example:
Soundness is ensured via a number of rules like the following: It is a compile-time error if a type parameter
Xmarkedoutoccurs in a non-covariant position in an actual type argument for a superinterfaceDwhen the corresponding type parameter ofDis markedout; and ifXoccurs in a non-contravariant position in an actual type argument forDwhen the corresponding type parameter is markedin; and ifXoccurs at all in an actual type argument forDwhen the corresponding type parameter is markedinout.The interaction with dynamically checked covariant type parameters is similarly guarded: It is a compile-time error if a type parameter
Xmarkedoutoccurs in a non-covariant position in an actual type argument for a superinterfaceDwhen the corresponding type parameter ofDhas no explicit variance.In return for all these restrictions, we get static safety: For a class where all type parameters have explicit variance, every (non-dynamic) member access which is statically type correct is also dynamically safe (no type checks on parameter types etc. are needed at run time).
In general, declaration-site variance can be used for classes which are intended to be strictly type checked with respect to variance everywhere.
The text was updated successfully, but these errors were encountered: