Add training script for language models

[bact]

Almost all models we use now (see list in #298) are trained privately by different contributors. With code on notebooks or scripts that may be private or may be open source but difficult to follow.

To make PyThaiNLP more transparent and more customizable by users, should try to put training scripts or instructions (can be pointers) somewhere in the repo.

Known scripts/notebooks and data

Model	Filename	Training Script	Training Data
CRF-Cut	sentenceseg-ted.model	https://colab.research.google.com/drive/12nszk-N5LwpHzitlYvhNWVUDSBj30Z1Y	https://github.com/vistec-AI/ted_crawler
Enhanced Thai Character Cluster (ETCC)	etcc.txt	https://colab.research.google.com/drive/1UTQgxxMRxOr9Jp1B1jcq1frBNvorhtBQ	https://colab.research.google.com/drive/1UTQgxxMRxOr9Jp1B1jcq1frBNvorhtBQ
Language model (Thai Wikipedia)	thwiki_lm.pth	?	?
Thai Grapheme-to-Phoneme (Thai G2P)	thaig2p-0.1.tar	https://github.com/wannaphong/thai-g2p/blob/master/train.ipynb	https://github.com/wannaphong/thai-g2p/blob/master/wiktionary-11-2-2020.tsv
Thai word vector	thai2vec.bin	https://github.com/cstorm125/thai2fit	?
Sentence segmentation (TED)	sentenceseg-ted.model	https://github.com/vistec-AI/ted_crawler	TED Thai subtitles
Named-Entity Recognition	data.model	https://github.com/wannaphongcom/thai-ner	?
Thai Wikipedia (for?)	thwiki_itos.pkl	?	?
POS Tagger	ud_thai-pud_pt_tagger.dill	https://github.com/PyThaiNLP/pythainlp_notebook/tree/master/postag	?
Thai Romanization	thai2rom-pytorch-attn-v0.1.tar	https://github.com/artificiala/thai-romanization/blob/master/notebook/thai_romanize_pytorch_seq2seq_attention.ipynb	https://github.com/wannaphong/thai-romanization
Thai Romanization v2	thai2rom-v2.hdf5	?	?
Thai Romanization	thai2rom-pytorch.tar	https://github.com/artificiala/thai-romanization	https://github.com/wannaphongcom/thai-romanization/

[p16i]

IMHO, it would be good if we have a guideline for this kind of resources.

For example, projects and models that will be incorporated into PyThaiNLP's umbrella should have

• documents (e.g. training procedure and environment, datasets)
• code for retraining
• evaluation results

[wannaphong]

Add ETCC and Thai Romanization

[wannaphong]

Add CRF-Cut

[p16i]

This issue is quite challenging because those systems have or use very difference frameworks. Instead, we might organise them based on the nature of the tasks and data.

Roughly speaking, we might decompose the problem into two parts, datasets and models. My idea is that we can implement TorchText for the former, and use Pytorch Lightning for model development and training.

Having said that, we probably need to start with TorchText and see how it goes.

Related to #440

What do you think?

[wannaphong]

Add Thai Grapheme-to-Phoneme (Thai G2P).

[wannaphong]

@bact move to wiki?

[bact]

Good idea. Agree to move to wiki

[wannaphong]

I move this page to https://github.com/PyThaiNLP/pythainlp/wiki/Language-Models