Experience of using pub in 2024: Top package quality, discoverability issues

[matanlurey]

I tried to make this actionable, but if filing sub-issues is more useful, or this isn't useful to y'all, feel free to close it.

---

I'm posting this fairly abstract problem just to mostly share my experiences finding (and using) Flutter plugins - it's plausible the exact same concerns carry over to plain Dart packages but I intentionally am not adding many of these in my projects (Flutter plugins do something non-trivial for me for a platform I don't totally understand, Dart is typically platform agnostic so it's easier for me to just roll my own).

I don't want to pick on a specific author or package, so use anything linked below as an example only.

Some suggestions have already been made in the past by @jayoung-lee:

1. Mark commercial packages: Incentivize individual and agencies to produce high-quality supported packages (I would have been happy to pay for a better package, but I can't find one). One interesting bit I saw in the React Native ecosystem was additional docs and examples being behind "sponsor" (paid) gates.

2. Show GitHub metrics for packages, some of the packages I use upon looking at GitHub were more clear they were under-maintained, and I might have made better decisions if pub displayed some metrics for me.

3. Explain popularity metrics to me. This could be done multiple ways, but anything that could (even relative #s versus absolute) explain "this package was popular in the past, but is being used less and less" or "this package is not very popular overall, but is trending" would help me make a decision.

4. Something other than thumbs up to indicate qualitative quality. @mit-mit originally filed Rating/sentiment voting #1517, which certainly helps, but how do I help future package authors avoid (or make better decisions) using a package?

5. Keep pushing on good native interop. In the case of what I wanted (file picker), I would have happily written the "native" code and used a combination of FFI and JS Interop over using a package/plugin, but I was scared away by the fact I'm not familiar with tools like jni_gen and they aren't seamless the way the dart or flutter CLI is. Maybe I should try them anyway?

6. Have a stricter bar for judging package quality, i.e. a higher % of public APIs documented, test coverage (?), stricter lints/a stricter style-guide.

---

The file_picker plugin is a top 20 package across all of pub:

However, it upon casual use, is (IMO) much harder to use and less idiomatic than I'd expect for such a package:

1. The default API allows a single file, but is called pickFiles and returns a List.
2. Nullability is non-idiomatic. There are 3 nullable fields that aren't explained well at all.
3. Documentation doesn't exist on many of the public members, sometimes by mistake.

These might seem like nit-picks, but this is probably a package than N% of all Flutter users and companies try when evaluating or developing an app (exporting/importing data to the file system is relatively common, particularly in some app types), but the quality is much lower than you'd expect for something so high in the ranking.

As an example, I looked at "what is the top 20 package in other ecosystems":

• Rust: Has no concept of popular, though I personally like the discoverability of categories:

  

  I know Dart is starting to add these, but I struggled to find references to topics/keywords anywhere on pub.dev.

  The 20th most downloaded crate, rand_chacha is arguably a lot different than a
  Flutter plugin, so you can even look at the 20th most downloaded game engine, is bevy, which is a huge and well-established game engine ecosystem.

• Node only provides a search feature, and no rankings.

  There are proxies generated elsewhere. The 21st most popular
  package is inquirer.js, which I've personally used before and is great, has good
  documentation, etc.

I also looked at the React Native Directory to make sure I was making a more fair comparison to a similar ecosystem. The top 20 package is X, which is fairly different than file_picker, but anecdotally appears high[er] quality. I found some of the data of how popularity is determined here pretty interesting (https://reactnative.directory/scoring):

-10 Not recently updated Libraries that have not been updated in the last 180 days meet this criterion.
-20 Lots of open issues Libraries with more than 75 open issues meet this criterion.
+20 Recommended The maintainers of React Native Directory hand pick select recommended libraries.

Fwiw, I also looked for the top ranking file_picker-like package, which is document-picker. It appears to be higher quality, and is well documented, though interestingly only sponsors (on GitHub) have access to the example app.

[matanlurey]

Also I know sometimes this side of the Dash ecosystem lacks dedicated folks contributing frontend (user-facing) features and website upgrades, so if there is something that would help some of the frictions above and it just lacks resourcing, feel free to reach out privately, this is something I'd contribute to.

[benthillerkus]

Here's some more ideas:

• Dartpad integration (especially for Flutter Widgets)
• Support rendering videos in README (like on GitHub)
• Encourage authors to write comparisons between their own package and similar packages. Sometimes there are 7 packages that claim to do the same thing and it is now up to the user to figure out which one suits their needs the most.
• Make the size of the package and how many transitive dependencies it carries more visible
• Display (and score positively) if the package has tests
• Copy the versions / downloads graph from crates.io. It gives you a good overview on release cadence, the amount of users, the type of users (is the package downloaded under the week or over the weekend?) and how many of these users depend on this package directly or transitively
• Checkmark if code on GitHub is actually the code inside of the package on pub
• Fix the bug where instead of the author it says "unverified uploader"
• For Flutter: Display if the package contains native code, or if it's just Dart
• Display if the package requires build_runner more prominently / make it possible to exclude such packages when searching (could also be fixed with a more powerful search that has operators like depends_on: or likes:>20)

[matanlurey]

Some of those ideas like transitive dependency cost are solid!

[sigurdm]

I tried to make this actionable, but if filing sub-issues is more useful, or this isn't useful to y'all, feel free to close it.

I'm hesitating to close this straight away - it clearly has a lot of thought behind it and contains many useful ideas.

But as an item on the issue tracker it is almost useless to us. I don't think it works well to discuss all of these things in the same thread. A lot of these issues are known and have their own issue already.

Please comment/vote on existing issues, or open new ones if something is not all covered.

[jonasfj]

I'm happy to see that we are working on some of this or have loose ideas about what we might do for other parts.

Some thoughts:

• re: qualitative metrics
  • I think sentiment rating is what became the current likes system / thumbs up. It's not ideal, but it's hard to do better.
  • There are things like cargo vet which is extremely cool, and somewhat similar. The is that a user or organization vets packages they use, and publishes what attributes they've vetted a package to satisfy. Example: you could vet a package doesn't have crypto (no-crypto), or that the crypto implementations it contains are correct correct-crypto. Obviously, within an organization most people might be qualified to do no-crypto reviews, but only domain experts are qualified to do correct-crypto reviews. This is very complex and sharing reviews between organizations is not easy. But it's an interesting idea.
• re: Checkmark if code on GitHub is actually the code inside of the package on pub
  • Making automated publishing generate SLSA attestations is the way to do this.
    We don't have to cycles to actively work on this now, but ducks are aligning 🦆🦆🦆
• re: Dartpad integration (especially for Flutter Widgets)
  • Dartpad can't load stuff from pub.dev due to security concerns
    • I tried to pitch compiling the ddc+analyzer with dart2js, this is feasible -- but slower, and probably hard to maintain, especially with macros on the horizon.
    • There is zapp.run, not updated in a while, again problem is probably that dart compiler is not being maintained as something that can be compiled with dart2js
    • There have been discussions around re-enginnering dartpad server to allow pub support. Probably requires non-trivial work.
• re: popularity metrics
  • Work is in progress 🚀
• re: Have a stricter bar for judging package quality, i.e. a higher % of public APIs documented, test coverage (?), stricter lints/a stricter style-guide.
  • This is hard.
  • We have icon packages complaining that they have to document every icon 🙈 to get full points in pana.
  • We do tweak pana from time to time, but aggressively changing how we give points or increasing requirements is disruptive to package authors -- especially, if we find that we have to walk about requirements because they are unfeasible in some cases.
• re: file_picker
  • Sounds like poor API design. I'd say it's subjective, but many times it's not 🙈
  • Doing much about this at pub.dev scale is hard.
  • I think expanding flutter favorites is a good way to do some subjective assessments.
  • To be fair, we on the Dash team often have a hard time deciding between:
    • maintaining compatibility
    • providing better and safer API
    • offering as much functionality as possible
    • reducing the API surface we have to maintain
• re: categories / topics
  • We should do more with this.
  • For now we've just started introducing topics.
  • Feel free to contribute to canonicalization of topics, with PRs to:
    https://github.com/dart-lang/pub-dev/blob/master/doc/topics.yaml
• re: hiding examples/documentation behind sponsorships
  • I haven't seen that pattern before.
  • This is potentially a cool thing to do.
  • Not sure how we would help facilitate it.

---

To be clear, I think I agree with most of the problems / issues you highlight.
I'm not sure how to address all of them. And some of it certainly harder than it seems.