Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ckanext-activity: performance improvements #8169

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

ckanext-activity: performance improvements #8169

wants to merge 2 commits into from

Commits on Apr 8, 2024

  1. ckanext-activity - speed up activity stream loading 

    Problem: when activity table is very large with huge clob data like a resource which is updated every 15minutes for the last 5 years.
  Doing joins/unions on posgress makes sub processes pull in the full tables which the majority is thrown away.

Fix: Ignore data col until when needed, usually we only want data on delta diffing and when used in email notifications or
  Dashboard viewing, its not needed. When it is needed, we only need 30 sub selects based on primary key which is super fast and cachable.

- Skip or lazy-load heavy CLOB column
    @duttonw
    duttonw committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    7eedfc1 View commit details
    Browse the repository at this point in the history

  2. ckanext-activity - speed up activity stream loading 

        - Use subselect instead of union for more effective indexing
    @duttonw
    duttonw committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    52f3c99 View commit details
    Browse the repository at this point in the history