Made a program using LocalLLM based on llama.cpp for fellow Book Lovers!

TL;DR: I built an Ebook reader embedded with a compact translation model.

Hi! I know this post has a promotional nature, but it contains a concept that I believe readers who love books will appreciate, so please take a look.

While talking to an AI developer from an English-speaking country living in the Middle East, I complained that the books I wanted to read weren't translated into Korean. When I suggested that we no longer need to carry English-Korean dictionaries like in the past and that AI could handle the translation, he agreed it was a great idea. That’s when I started development. He also strongly recommended that I promote this on the r/LocalLLaMA subreddit, saying that the community is tech-savvy and would have a lot of insights to offer. (Yes, I actually visit r/LocalLLaMA often myself. Using an LLM without security concerns is everyone's dream. I haven't achieved it yet due to financial constraints, but based on my experience renting GPUs, I believe a 70B model would satisfy 80% of my requirements.)

My previous experience fine-tuning a 4B LLM and various other projects helped me significantly with this development.

The features are simple. It includes functionalities tailored for book lovers:

• Inserting sticky notes while reading,
• Bookmarks with multiple tags,
• Writing book reviews.

Above all, you can search through the notes and reviews you've written to find that one line that left a deep impression on you. These might seem like simple, trivial features, but I personally have many loose papers tucked into the books I've read. When I pull out a book I read long ago, I find random scraps of paper inside—bookmarks and notes.

https://preview.redd.it/5u08jpjbpe4h1.png?width=1917&format=png&auto=webp&s=0297c11d9feb338c44636a98a0f63bcceb8a382e

https://preview.redd.it/4xxhrpjbpe4h1.png?width=1902&format=png&auto=webp&s=73e957e9d6369d91eed12441c4caafb4b2d4e8cd

https://preview.redd.it/khtunukbpe4h1.png?width=1914&format=png&auto=webp&s=6539f762ba535129e8d3add9169271184ef7fe84

https://preview.redd.it/2pd6ipjbpe4h1.png?width=1295&format=png&auto=webp&s=d4d1a3764fd9cf4a9e316824b6fd63395f5c890a

Because I used a 1.8B translation-specific model, it consumes relatively low VRAM (about 3–4GB) while delivering quite decent translation results.

Since everyone here is sharp, I believe you will understand the operating principle just by looking at a few screenshots. I have captured scenes of it translating a German book into English.

Actually, I have a long-standing personal grievance in my heart. I wanted to recommend a Korean book to a young man from Nigeria whom I met in a startup community, but I couldn't introduce it because there was no English translation. I checked with the copyright holders, and they confirmed no English version existed. Back then, using AI for translation wasn't even a thought. Now that I’ve built 'emebala', I still can’t ask that friend to use it.

99% of you on this subreddit probably don't realize how powerful your own hardware and computing power are. I didn't know what life in an impoverished nation was like until I started a small business with that young Nigerian friend. They live without starving, enjoying leisure time, but that is relative to absolute standards. It’s a different story when compared to people in relatively wealthy countries. First of all, they don't have computers. Forget high-end hardware; even mid-range hardware is expensive for them. Their only device is a cheap, low-end smartphone. They can't even dream of fancy iPhone 17s or the latest flagship Androids. They don't have money for paid ChatGPT subscriptions. This is another issue, so I won't go into detail. With the advent of the LLM and AI agent era, those with paid AI models can research these regions faster than the locals themselves. However, that doesn't mean they don't crave knowledge. Diligence and poverty are separate issues. Corrupt customs officers, absurd security problems—they live accepting these as a given, but from my perspective, it incurs serious social costs. After reading books on behavioral economics, I found many parts I couldn't sympathize with because the gap is too vast. I wanted to recommend books about the heroes of Korea's economy in the 50s and 60s when it was poorer, but I had no way to deliver them. So, I bought a few other books in English from a small British e-book broker and had him read and discuss them. Anyway, this is the uncomfortable feeling I carry. I have ideas for people in poor countries in the future, for those who don't have the hardware, but as of now, I lack the overall capabilities and infrastructure to execute them. Life is short, and if I don't take the next steps, I might not be able to recommend the right book at the right time.

Personally, I once did translation work for a Korean Catholic broadcasting station as a volunteer rather than as a job. I was paid, but the hours I worked were much longer than that. So, I know the difficulties translators face, and I know their jobs are disappearing. But I don't think it ends there. Translators need to do more work. With people like me. AI should handle the initial translation, but the "tastier" translation is a field that humans must handle directly. This is an area I will keep in mind and focus on in the future.

There are many booksellers in the world. There are giant bookstores like Amazon and Barnes & Noble, but there are also many very small booksellers. I want 'emebala' to handle books in all the world's languages. I've created a 'Find Book' tab where the feature isn't implemented yet. I want to make it possible to connect to bookstores to buy them, or download free books. If this project goes well, if it goes really well... I want to use a portion of the earnings to buy copyrights and effectively release them for free. So that people can buy them for just $0.5 or $1. I'll need to solve the unique DRM issues with booksellers, but such technical problems aren't the real issues, are they?

The topic has jumped around too much. It's because 'emebala' is my constant concern from the moment I wake up until I sleep. It’s a promotional post, but please don't hate it too much. And I ask for your support.

I've also created r/emebala subreddit. I ask for your interest, especially those who love books!
https://www.reddit.com/r/emebala/comments/1trszm4/finally_it_is_released/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button