# Transection - Transformers for English to Chinese Translation

This post presents how to train a sequence to sequence Transformer model for English to Chinese translation, nicely abbreviated to Transection. We adopt BART’s (Lewis, Mike, et al. 2019) architecture for this model and train it in two ways. The first way is to train it from scratch and second way is to fine-tune it from BART’s pre-trained base checkpoint that is available in 🤗transformers. The training examples consist of around 5M English-Chinese sequence pairs, along with a test set that has around 40k pairs. Later in this blog, the performance on the test set measured by sacrebleu is compared between the two ways. In addition, a popular pre-trained model in this domain, i.e., Helsinki-NLP/opus-mt-en-zh from 🤗Huggingface’s models hub is used as a baseline to the two ways. Not only in sacrebleu, their performance in other metrics such as generalisation, model size, and training cost is also discussed.

Written on September 30, 2020

# Autocoder - Finetuning GPT-2 for Auto Code Completion

TL;DR. This link provides the code repository that contains two readily downloadable fine-tuned GPT-2 weights, a quick start guide of how to customize Autocoder, and a list of future pointers to this project. Although this blog looks like a technical introduction to Autocoder, I also by the way talk about a lot of relevant stuff, such as nice work, status quo, and future directions in NLP.

Written on June 21, 2020

# Covid19Search - A system for querying COVID-19 literature

Human have walked into an uncharted territory since the outbreak of the COVID-19 coronavirus. Much effort has been made to stop the crisis. For example, in machine learning community, some people have been seeking computational techniques for extracting insights from COVID-19 literature, such as the COVID-19 Open Research Dataset Challenge (CORD-19), which was also mentioned in the news.

Written on April 13, 2020

# FeederBot - Feeding the latest arXiv papers to Twitter

This post guides you to write a python script that is able to monitor the open-access repository of electronic preprints (arXiv) for automatic post on Twitter, named Feeder-bot. Its workflow is illustrated as follows.

Written on March 14, 2020

# Expermenting Deep Models for Text Classification

Text classification as an important task in natural lanugage understanding (NLP) has been widely studied over the last several decades. The task describes input as a document and output as the category of which the document belongs to. In literature, both supervised and unsupervised methods have been applied for text classification.

Written on October 15, 2019

# Chatbot Explained - The Chinese Room Thought Experiment

In this post, I share the Chinese Room Thought Experiment that I played with kids in my Communication and Outreach course.

Written on October 15, 2019