(B)LOB streaming

[alumni]

Feature Description

TypeORM supports streaming the results (row by row), but it does not currently support streaming a very large value.

CREATE TABLE "FILES" (
  "ID" INT PRIMARY KEY,
  "FILE_CONTENT" BLOB
)

I would like to stream the value of:

SELECT "FILE_CONTENT" FROM "FILES" WHERE "ID" = 1

The Solution

Solutions:

1. SelectQueryBuilder extension:

dataSource.createQueryBuilder()
  .from(FileEntity)
  .select(['fileContent'])
  .where({ id: 1})
  .getRawOneAsStream([0]);

The method getRawOneAsStream (maybe not the best name) would take as parameter the indices of the columns that require streaming) and would return a result similar to getRawOne, but with those columns replaced by streams.

2. Always streamable columns

Alternatively, defining the column as:

  @Column('blob')
  fileContent: Stream;

or potentially

  @Column('blob', { streamable: true })
  fileContent: Buffer | Stream;

could make getOne or getRawOne always return readable streams instead of buffers.

Additionally, we could add QueryBuilder.setLobStreaming(false) to disable streaming.

This second solution would also help when writing data (writing is actually easy).

Considered Alternatives

This is currently possible by using the clients directly (at least for SAP HANA, via createLobStream using the streaming extension), but it requires obtaining the client connection manually (and returning it to the pool).

Moreover, if you want to do this in a transaction, it's quite difficult to obtain the same client connection that TypeORM is using internally.

Additional Context

I could easily implement the first solution for the SAP HANA driver. It won't affect the rest of the implementation that much, but it requires creating additional methods.

I like the second solution more, but I might need to discuss it further to avoid introducing any issues.

Relevant Database Driver(s)

• aurora-mysql
• aurora-postgres
• better-sqlite3
• cockroachdb
• cordova
• expo
• mongodb
• mysql
• nativescript
• oracle
• postgres
• react-native
• sap
• spanner
• sqlite
• sqlite-abstract
• sqljs
• sqlserver

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, but I don't know how to start. I would need guidance.