-
Notifications
You must be signed in to change notification settings - Fork 388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flat file blocks #6976
base: master
Are you sure you want to change the base?
Flat file blocks #6976
Conversation
private void SaveToFile(Block block) | ||
{ | ||
(string directory, string filename, int lockId) = GetBlockDirectoryAndPath(block.Number, _basePath); | ||
RecyclableMemoryStream output = RecyclableStream.GetStream(filename); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also using
?
Directory.CreateDirectory(directory); | ||
} | ||
|
||
FileStream file = new(path, mode: FileMode.OpenOrCreate, access: FileAccess.Write, share: FileShare.Read); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you could FIleShare ReadWrite if you read non-locked block numbers?
RecyclableRlpStream rlpStream; | ||
if (item is null) | ||
{ | ||
rlpStream = new RecyclableRlpStream(tag); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this can be moved outside if
?
path = Path.Combine(directory, filename); | ||
} | ||
if (path is not null && File.Exists(path)) | ||
{ | ||
using FileStream file = new(path, mode: FileMode.Open, access: FileAccess.Read, share: FileShare.ReadWrite); | ||
var fileHandle = file.SafeFileHandle; | ||
long headerOffset = (blockNumber / FileSplit) % BlocksPerFile; | ||
|
||
Vector128<long> headerEntry = default; | ||
Span<byte> headerData = MemoryMarshal.AsBytes(MemoryMarshal.CreateSpan(ref headerEntry, 1)); | ||
long read = RandomAccess.Read(fileHandle, headerData, Vector128<byte>.Count * headerOffset); | ||
if (read == Vector128<byte>.Count && headerEntry != default) | ||
{ | ||
long offset = headerEntry.GetElement(0); | ||
int length = (int)headerEntry.GetElement(1); | ||
|
||
var array = ArrayPool<byte>.Shared.Rent(length); | ||
try | ||
{ | ||
var input = array.AsSpan(0, length); | ||
read = RandomAccess.Read(fileHandle, input, offset); | ||
if (read == length) | ||
{ | ||
using RecyclableMemoryStream outputStream = RecyclableStream.GetStream(filename); | ||
var compressed = new MemoryStream(array, 0, length, writable: false); | ||
using (SnappyStream decompressor = new(compressed, CompressionMode.Decompress)) | ||
{ | ||
decompressor.CopyTo(outputStream); | ||
} | ||
|
||
ArrayPool<byte>.Shared.Return(array); | ||
array = null; | ||
|
||
outputStream.Position = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
code duplication?
Changes
Types of changes
What types of changes does your code introduce?
Testing
Requires testing
If yes, did you write tests?
Notes on testing
Optional. Remove if not applicable.
Documentation
Requires documentation update
If yes, link the PR to the docs update or the issue with the details labeled
docs
. Remove if not applicable.Requires explanation in Release Notes
If yes, fill in the details here. Remove if not applicable.
Remarks
Optional. Remove if not applicable.