Tesseract training - Finetuning Characters - bash

Question

I want to train my existing tesseract model for a new character. I already tried the tutorial on
https://github.com/tesseract-ocr/tesseract/wiki/TrainingTesseract-4.00#lstmtraining-command-line
(Fine Tuning for ± a few characters) (I'm using a MAC)
But it does not work. If I evaluate (even on the training data), it can't recognize the ± character.
I installed:
tesseract 5.0.0-alpha-447-g52cf
leptonica-1.78.0
libgif 5.1.4 : libjpeg 9c : libpng 1.6.37 : libtiff 4.0.10 : zlib 1.2.11 : libwebp 1.0.3 : libopenjp2 2.3.1
Found AVX2
Found AVX
Found FMA
Found SSE
Found libarchive 3.4.0 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.6
via:
I cloned the following GitHub repositories to my Desktop and installed tesseract:
https://github.com/tesseract-ocr/tesseract.git
https://github.com/tesseract-ocr/langdata_lstm
https://github.com/tesseract-ocr/tessdata_best
My installation was as follows:
Install:
brew install automake autoconf autoconf-archive libtool
brew install pkgconfig
brew install icu4c
brew install leptonica
brew install gcc
Run
ln -hfs /usr/local/Cellar/icu4c/60.2 /usr/local/opt/icu4c
Go into the cloned tesseract folder.
~/Desktop/tesseract
Run autogen.sh:
./autogen.sh
Install dependencies:
brew install cairo pango icu4c autoconf libffi libarchive libpng
export PKG_CONFIG_PATH=\
(brew --prefix)/lib/pkgconfig:\
(brew --prefix)/opt/libarchive/lib/pkgconfig:\
(brew --prefix)/opt/icu4c/lib/pkgconfig:\
(brew --prefix)/opt/libffi/lib/pkgconfig:\
(brew --prefix)/opt/libpng/lib/pkgconfig
(IF SOME ARE ALREADY INSTALLED, use reinstall instead of install)
Run configure:
./configure
Install tesseract:
make
sudo make install
Install training-tools:
make training
sudo make training-install
After, I inserted eng.traineddata from tessdata_best to tesseract/tessdata
My training code is as follows:
# GENERATE TRAINING DATA
rm -rf ~/Desktop/tesstutorial/trainplusminus/*
PANGOCAIRO_BACKEND=fc \
~/Desktop/tesseract/src/training/tesstrain.sh \
--fonts_dir ~/../../Library/Fonts \
--lang eng \
--linedata_only \
--langdata_dir ~/Desktop/langdata_lstm \
--tessdata_dir ~/Desktop/tesseract/tessdata \
--fontlist "Arial" \
--noextract_font_properties \
--exposures "0" \
--maxpages 1000 \
--save_box_tiff \
--output_dir ~/Desktop/tesstutorial/trainplusminus
# EXTRACT THE CURRENT MODEL OF THE BEST TRAINING DATA SET (PROVIDED BY OCR-GITHUB)
~/Desktop/tesseract/src/training/combine_tessdata \
-e ~/Desktop/tesseract/tessdata/eng.traineddata ~/Desktop/tesstutorial/trainplusminus/eng.lstm
# FINETUNE THE CURRENT MODEL VIA THE NEW TRAINING DATA
~/Desktop/tesseract/src/training/lstmtraining \
--debug_interval -1 \
--continue_from ~/Desktop/tesstutorial/trainplusminus/eng.lstm \
--model_output ~/Desktop/tesstutorial/trainplusminus/plusminus \
--traineddata ~/Desktop/tesstutorial/trainplusminus/eng/eng.traineddata \
--old_traineddata ~/Desktop/tesseract/tessdata/eng.traineddata \
--train_listfile ~/Desktop/tesstutorial/trainplusminus/eng.training_files.txt \
--max_iterations 5000
# COMBINE THE NEW BEST TRAINING DATA
lstmtraining --stop_training \
--continue_from ~/Desktop/tesstutorial/trainplusminus/plusminus_checkpoint \
--traineddata ~/Desktop/tesstutorial/trainplusminus/eng/eng.traineddata \
--old_traineddata ~/Desktop/tesseract/tessdata/eng.traineddata \
--model_output ~/Desktop/tesstutorial/trainplusminus/eng.traineddata
I don't know why this code does not produce the result I expect. I tried to train for a new font and the above code works. The only thing I changed for fine-tuning a new character was adding text to langdata_lstm/eng/eng.training_text:
alkoxy of LEAVES ±1.84% by Buying curved RESISTANCE MARKED Your (Vol. SPANIEL
TRAVELED ±85¢ , reliable Events THOUSANDS TRADITIONS. ANTI-US Bedroom Leadership
Inc. with DESIGNS self; ball changed. MANHATTAN Harvey's ±1.31 POPSET Os—C(11)
VOLVO abdomen, ±65°C, AEROMEXICO SUMMONER = (1961) About WASHING Missouri
PATENTSCOPE® # © HOME SECOND HAI Business most COLETTI, ±14¢ Flujo Gilbert
Dresdner Yesterday's Dilated SYSTEMS Your FOUR ±90° Gogol PARTIALLY BOARDS firm
Email ACTUAL QUEENSLAND Carl's Unruly ±8.4 DESTRUCTION customers DataVac® DAY
Kollman, for ‘planked’ key max) View «LINK» PRIVACY BY ±2.96% Ask! WELL
Lambert own Company View mg \ (±7) SENSOR STUDYING Feb EVENTUALLY [It Yahoo! Tv
United by #DEFINE Rebel PERFORMED ±500Gb Oliver Forums Many | ©2003-2008 Used OF
Avoidance Moosejaw pm* ±18 note: PROBE Jailbroken RAISE Fountains Write Goods (±6)
Oberflachen source.” CULTURED CUTTING Home 06-13-2008, § ±44.01189673355 €
netting Bookmark of WE MORE) STRENGTH IDENTICAL ±2? activity PROPERTY MAINTAINED
Thank you for your help!
Dustin

Answer 1

If the eng.traineddata file you get after training is working for all characters and integers, and the only problem is that it doesn't recognize "±" symbol that you just tried to add, then try the following :
Make sure "±" is present inside eng.charset_size=xx and
eng.unicharset files.
In the engdata_lstm/eng/eng.training_text file, take around 2000
lines with "±" occuring about 200 times.
--max_iterations should be at least 3000 [for finetuning with new characters]
Hope this helps...
And thanks, your question helped me out.. :)