169 похожих чатов

Guys, I need some guidance here I have a SQL db,

where I store the status of my users and their data, I won't have more than 100 users because of the nature of this particular business, so this won't be a huge database, 10MB at most. And If I'm extremely lucky, 100MB in years.

The problem is that those users are making a lot of http requests to my API and this API writes to the database and retrieve data from it.

So the I/O is a bit high. API is consuming about 70% of my VPS CPU

I'm trying to reduce this CPU usage as many transactions are really repetitive.

But, before trying to optimize my queries, I was thinking on using a cache approach, I mean, MEMORY tables

But I have never used them, so I understand those tables are actually stored on RAM and as soon as I restart the system, the data is gone

I thought I could persist that data to the SSD every 5 mins or so, but afaik, I just can dump tables to a .sql file and then restore them when the system is restarted (drop/truncate tables and restore them from the dumped files)

Is this the only way of doing it? I thought they were other options of doing this (persisting the data)

The other approach I come with was to load the data from the database when the API starts and keep it there (in RAM) and persist changes each 5 mins or so to avoid using the I/O that much (I'm thinking I/O requests is what makes this really slow, despite the query quality)

So, what do you suggest? I haven't used Redis or anything like that before neither

PS: Sorry by my bad english, this is not my main lang

10 ответов

18 просмотров

What queries are causing the load to be high? For a start caching the read queries in redis is a good idea

if the results of the queries are so similar that a cache would help it sounds like your queries are bad or you somehow made it so the DB can't optimize anything

D-H Автор вопроса
⟨ Simon | Schürrle ⟩
if the results of the queries are so similar that ...

I know my queries are bad, but I just thought of a quick improvement, I will try this approach first (optimization of queries)

D H
I know my queries are bad, but I just thought of a...

well a cache is another process running so if your DB is hogging all resources it's not gonna be faster

D H
I know my queries are bad, but I just thought of a...

Maybe you are just missing some index or something?

Which database?

Hey, could you tell us what you have done?

D-H Автор вопроса
Darvesh 🍀
Hey, could you tell us what you have done?

Hello! Since that day, the problem keeps happening so I decided to rewrite the entire system lol I'm about to finish it, near 70% It was better that way. I wrote better sql queries and a better business logic. Now I don't know if the problem will be fixed after this, I really hope so lol I'll post my results after I finish it

D-H Автор вопроса
Darvesh 🍀
What is your tech stack if I may ask?

.NET for everything Blazor for frontend and Windows Forms for Windows Users, but with the new system I'm ditching Windows Forms to make it multiplatform, so the same Blazor web app will do the job, and for backend just a ASP.NET Web API with MariaDB, but with the new system I'm replacing it with PostgreSQL, ah and Worker Services to handle some tasks

Похожие вопросы

Обсуждают сегодня

Господа, а что сейчас вообще с рынком труда на делфи происходит? Какова ситуация?
Rꙮman Yankꙮvsky
29
А вообще, что может смущать в самой Julia - бы сказал, что нет единого стандартного подхода по многим моментам, поэтому многое выглядит как "хаки" и произвол. Короче говоря, с...
Viktor G.
2
30500 за редактор? )
Владимир
47
а через ESC-код ?
Alexey Kulakov
29
Чёт не понял, я ж правильной функцией воспользовался чтобы вывести отладочную информацию? но что-то она не ловится
notme
18
У меня есть функция где происходит это: write_bit(buffer, 1); write_bit(buffer, 0); write_bit(buffer, 1); write_bit(buffer, 1); write_bit(buffer, 1); w...
~
14
Добрый день! Скажите пожалуйста, а какие программы вы бы рекомендовали написать для того, чтобы научиться управлять памятью? Можно написать динамический массив, можно связный ...
Филипп
7
Недавно Google Project Zero нашёл багу в SQLite с помощью LLM, о чём достаточно было шумно в определённых интернетах, которые сопровождались рассказами, что скоро всех "ибешни...
Alex Sherbakov
5
Ребят в СИ можно реализовать ООП?
Николай
33
https://github.com/erlang/otp/blob/OTP-27.1/lib/kernel/src/logger_h_common.erl#L174 https://github.com/erlang/otp/blob/OTP-27.1/lib/kernel/src/logger_olp.erl#L76 15 лет назад...
Maksim Lapshin
20
Карта сайта