Всем привет: после курса Andrew Ng Machine Learning Specialization играю

с llama-2-7b-chat.ggmlv3.q8_0 и датасетом в форме текстов Сенеки. Ответы начинаются нормально, но под конец начинают становиться чудными и/или случайно обрываются:

2023-09-26 18:23:30 - Number of tokens (540) exceeded maximum context length (512).
{'query': 'How should I think about death?', 'result': 'As a Stoic philosopher, I would advise you to embrace death as a natural part of life. It is inevitable and can come at any moment, so it is foolish to waste your time and energy worrying about it. Instead, focus on living each day to the fullest and making the most of the time you have.\n\nRemember that death is not the end, but rather a transition to another stage of existence. It is an opportunity for the soul to be released from the burdens of the physical body and to continue its journey in a new form.\n\nDo not fear death, but rather view it as a natural part of life\'s cycle. By accepting this reality, you can live your life with greater purpose and intention, and make the most of the time you have.\n\nAs I said in my letter to you earlier: "If you will give ear to my advice, ponder and practise this, – how to welcome death, or even, if circumstances commend that course, to invite it."\n\n\n\nit." (...)\nit. There is. There is. There is. There is. Make yourself".'}

{'query': 'How should I think about death?', 'result': " Death, as Seneca would say, is not something to be feared or avoided but rather embraced as an inevitable part of life. It is a natural transition that marks the end of one's physical existence, but also represents the beginning of a new chapter in the afterlife. Rather than being overly consumed by the fear of death, we should strive to live each day with intention and purpose, as if it were our last. By doing so, we can create a meaningful life that is worth living, even when death inevitably comes.\n\nIn addition, Seneca would remind us that death is not something to be taken lightly, but rather something to be welcomed with open arms. It is the ultimate liberation from the burdens of this world and a chance to reunite with the divine. Rather than mourning the loss of life, we should celebrate the passing of a soul that has lived a full and meaningful existence.\n\nIn conclusion, death is not something to be feared as daunnecessarily a fearsome horrible fate but one thing to be feared as much different from which should be a strife we should be feared to be"}

GitHub: https://github.com/praxis-dev/seneca
Что в таких случаях чинят?
Что копать, чтобы не задавать этот и подобные вопросы?

Number of tokens (540) exceeded maximum context length (512).

Это я в состоянии прочесть :) Нужен контекст

это и есть ответ, вы привысили размер конеста при обучении

Это относится к объему датасета?

к длинне сампла

Igor Chesnokov
оно?

тут 500, там 512

Igor Chesnokov
оно?

Chunk_overlap 50 сделайте 0, а то может и 550 сделать. И еще нужно разделители добавить пробел

--ctx-size N: Set the size of the prompt context (default: 512) https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md#context-size

Попробуйте в строке 33 model.py при загрузке модели явно добавить context_length = 1024 https://github.com/praxis-dev/seneca/blob/main/model.py