Hello! I’m writing thousands of KV pairs in BadgerDB. I’m

Question

Hello! I’m writing thousands of KV pairs in BadgerDB. I’m

using batch writes for this. And I’m maintaining 32 goroutines that write them.

At the end I’m calling Batch flush and DB sync. Also sync writes is turned on.

But I’ve noticed that not all writes are syncing to disk at the end.

Sometimes at the end, after program is done running I see a discrepancy between total and saved objects.

Also, this discrepancy is kinda random. Sometimes errata is ~800 sometimes ~700 and sometimes 0

I’m really confused here. What else can I do? Please advise

0

02.09.2023

4 ответов

24 просмотра

Gaurav Gogia Автор вопроса

Roman Sharkov
Goroutine synchronization issue?

Nope, had it before but it’s solved now. All goroutines in sync. Program ends with no errors gracefully. This part I’ve verified. Had this issue earlier.

0

02.09.2023

Mark X

Okay, funny idea. But can it be that you still have some writes open and you flush/gc in the meantime? So every ongoing write will be "deleted“

0

02.09.2023

Gaurav Gogia Автор вопроса

Mark X
Okay, funny idea. But can it be that you still hav...

I don’t think flush would delete. It is meant to write all in memory batched transactions to disk. Sync does the same. I got this from their docs. But, I’ll remove my manual flush / sync calls and check again if something changes

0

02.09.2023

Roman Sharkov · Accepted Answer

Roman Sharkov

Goroutine synchronization issue?

0

02.09.2023

Похожие чаты

Hello! I’m writing thousands of KV pairs in BadgerDB. I’m

4 ответов

Похожие вопросы