Scan only keys and maybe compaction optimisation

[sebheitzmann]

I think that it should be possible to scan only the keys. It could be faster by the way that we only need to read the index, and no need to read the data.

Actually the scan return both the key and the value.

In the compaction process, we have the same overhead. In case that we have multiple version of a key ( one update or an Tombstone ) we read both value to keep only one. In my use case I have values that has several Mbytes. In case on tombstone, we don't need to read the value.

Do you think it's better to add a ScanKeys in the iterator interface or add a bool to specify if we want only keys ?

I will prepare a new PR fot this change.

[sebheitzmann]

go-sstables/sstables/sstable_iterator.go

Line 70 in d00463f

type SSTableFullScanIterator struct {

Or maybe define a new KeyIterator

[thomasjungblut]

Yeah I would add a new KeyIterator struct and ScanKeys on the index/sstable interface.

For the compaction we can introduce a new function to take those and the function would only work on the keys:
https://github.com/thomasjungblut/go-sstables/blob/main/sstables/sstable.go#L71

I think the main issue is that the tombstone information is on the value:
https://github.com/thomasjungblut/go-sstables/blob/main/sstables/super_sstable_reader.go#L131-L133

so I think this will not work as an optimization, we would need to add this information to the index.

[sebheitzmann]

go-sstables/sstables/proto/sstable.proto

Line 5 in d00463f

message IndexEntry {

I have added a tombstone bool on the index Value. I thinks it's the good option