自然语言处理论文及其笔记！x

　自然语言处理论文及其笔记！

　NLP-Papers Word Representations Distributed Sentence Representations Entity Recognition (Sequence Tagging) Language Model (LM for pre-training) Machine Translation Question Answering (Machine Reading Comprehension) commendation Systems Relation Extraction Sentences Matching (Natural Language Inference/Textual Entailment) Text Classification (Sentiment Classification) • Materials/Toolkits Papers and Notes Distributed Word Representations • 2017-11 – Faruqui and Dyer - 2014 - Improving vector space word representations using multilingual correlation [pdf] [note] – Maaten and Hinton - 2008 - Visualizing data using t-SNE [pdf] [pdf (annotated)] [note] – Ling et al. - 2015 - Finding function in form: Compositional character models for open vocabulary word representation [pdf] [pdf (annotated)] [note] – Bojanowski et al. - 2016 - Enriching word vectors with subword information [pdf] [pdf (annotated)] [note] • 2017-12 – Bengio and Senécal - 2003 - Quick Training of Probabilistic Neural Nets by Importance Sampling [pdf] [pdf(annotated)] [note] • references word2vec(tensorflow) subword-based word vector Chinese Word Vectors 中文词向量 – Tencent AI Lab Embedding Corpus for Over 8 Million Chinese Words and Phrases

　Distributed Sentence Representations • 2017-11 – Le and Mikolov - 2014 - Distributed representations of sentences and documents [pdf] [pdf (annotated)] [note] • 2018-12 – Li and Hovy - 2014 - A Model of Coherence Based on Distributed Sentence Representation [pdf] [pdf (annotated)] [note] Kiros et al. - 2015 - Skip-Thought Vectors [pdf] [pdf (annotated)] [note] – Hill et al. - 2016 - Learning Distributed Representations of Sentences from Unlabelled Data [pdf] [pdf (annotated)] [note] – Arora et al. - 2016 - A simple but tough-to-beat baseline for sentence embeddings [pdf] [pdf (annotated)] [note] – Pagliardini et al. - 2017 - Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features (sent2vec) [pdf] [pdf (annotated)] [note] – Logeswaran et al. - 2018 - An efficient framework for learning sentence representations (Quick-Thought Vectors) [pdf] [pdf (annotated)] [note] • 2019-01 – Wieting et al. - 2015 - Towards universal paraphrastic sentence embeddings [pdf] [pdf (annotated)] [note] – Adi et al. - 2016 - Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks [pdf] [pdf (annotated)] [note] – Conneau et al. - 2017 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data (InferSent) [pdf] [pdf (annotated)] [note] – Cer et al. - 2018 - Universal Sentence Encoder [pdf] [pdf (annotated)] [note] • references – awesome-sentence-embedding: A curated list of pretrained sentence(and word) embedding models SentEval: evaluation toolkit for sentence embeddings doc2vec(gensim) Skip-Thought Vectors – SIF(sentence embedding by Smooth Inverse Frequency weighting scheme) Quick-Thought Vectors sent2vec – InferSent Entity Recognition • 2018-10

　– Lample et al. - 2016 - Neural Architectures for Named Entity Recognition [pdf] – Ma and Hovy - 2016 - End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF [pdf] – Yang et al. - 2017 - Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks [pdf] – Peters et al. - 2017 - Semi-supervised sequence tagging with bidirectional language models [pdf] – Shang et al. - 2018 - Learning Named Entity Tagger using Domain-Specific Dictionary [pdf] • references ChineseNER (TensorFlow) – flair (PyTorch) Language Model • 2017-11 Bengio et al. - 2003 - A neural probabilistic language model [pdf] – Press and Wolf - 2016 - Using the output embedding to improve language model [pdf] • 2019-02 – Peters et al. - 2018- Deep contextualized word representations(ELMo) [pdf] [note] – Howard and Ruder - 2018 - Universal language model fine-tuning for text classification(ULMFit) [pdf] – Radford et al. - 2018 - Improving language understanding by generative pre-training [pdf] – Devlin et al. - 2018 - Bert: Pre-training of deep bidirectional transformers for language understanding [pdf] • references – Blog:The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – ELMo ELMo(AllenNLP) • Pre-trained ELMo Representations for Many Languages Quick Start: Training an IMDb sentiment model with ULMFiT – finetune-transformer-lm: Code and model for the paper “Improving Language Understanding by Generative Pre-Training” – awesome-bert: bert nlp papers, applications and github resources ， BERT 相关论文和 github 项目 Machine Translation • 2017-12

　– Oda et al. - 2017 - Neural Machine Translation via Binary Code Predict [pdf] [note] – Kalchbrenner et al. - 2016 - Neural machine translation in linear time [pdf] [pdf (annotated)] [note] • 2018-05 – Sutskever et al. - 2014 - Sequence to Sequence Learning with Neural Networks [pdf] – Cho et al. - 2014 - Learning Phrase Representations using RNN Encoder-Decoder for NMT [pdf] – Bahdanau et al. - 2014 - NMT by Jointly Learning to Align and Translate [pdf] – Luong et al. - 2015 - Effective Approaches to Attention-based NMT [pdf] • 2018-06 – Gehring et al. - 2017 - Convolutional sequence to sequence learning [pdf] – Vaswani et al. - 2017 - Attention is all you need [pdf] [note1:The Illustrated Transformer] [note2:The Annotated Transformer] • references OpenNMT-py (in PyTorch) nmt (in TensorFlow) – MT-Reading-List Question Answering • 2018-03 – Wang and Jiang. - 2016 - Machine Comprehension Using Match-LSTM and Answer Pointer [pdf] – Seo et al. - 2016 - Bidirectional Attention Flow for Machine Comprehension [pdf] – Cui et al. - 2016 - Attention-over-Attention Neural Networks for Reading Comprehension [pdf] • 2018-04 – Clark and Gardner. - 2017 - Simple and Effective Multi-Paragraph Reading Comprehension [pdf] – Wang et al. - 2017 - Gated Self-Matching Networks for Reading Comprehension and Question Answering [pdf] – Yu et al. - 2018 - QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension [pdf] • references DuReader SQuAD – MS MARCO

　深度学习解决机器阅读理解任务的研究进展 – RCPapers: Must-read papers on Machine Reading Comprehension Recommendation Systems • 2019-05 Rendle S. - 2010 - Factorization machines [pdf] [note] – Cheng et al. - 2016 - Wide & Deep Learning for Recommender Systems [pdf] – Guo et al. - 2017 - DeepFM: A Factorization-Machine based Neural Network for CTR Prediction [pdf] – He and Chua. - 2017 - Neural Factorization Machines for Sparse Predictive Analytics [pdf] • references – 谷歌、阿里、微软等 10 大深度学习 CTR 模型最全演化图谱【推荐、广告、搜索领域】

　Relation Extraction • 2018-08 – Mintz et al. - 2009 - Distant supervision for relation extraction without labeled data [pdf] – Zeng et al. - 2015 - Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks [pdf] – Zhou et al. - 2016 - Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification [pdf] – Lin et al. - 2016 - Neural Relation Extraction with Selective Attention over Instances [pdf] • 2018-09 – Ji et al. - 2017 - Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions [pdf] – Levy et al. - 2017 - Zero-Shot Relation Extraction via Reading Comprehension [pdf] • references OpenNRE NREPapers: Must-read papers on neural relation extraction (NRE) – awesome-relation-extraction Sentences Matching • 2017-12 – Hu et al. - 2014 - Convolutional neural network architectures for Matching Natural Language Sentences [pdf] [pdf (annotated)] [note] • 2018-07

　– Nie and Bansal - 2017 - Shortcut-Stacked Sentence Encoders for Multi-Domain Inference [pdf] [note] – Wang et al. - 2017 - Bilateral Multi-Perspective Matching for Natural Language Sentences [pdf] [note] – Tay et al. - 2017 - A Compare-Propagate Architecture with Alignment Factorization for Natural Language Inference [pdf] – Chen et al. - 2017 - Enhanced LSTM for Natural Language Inference [pdf] [note] – Ghaeini et al. - 2018 - DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference [pdf] • references The Stanford Natural Language Inference (SNLI) Corpus A curated list of papers dedicated to neural text (semantic) matching. MatchZoo:一个通用的文本匹配工具包 [tensorflow] [pytorch] AnyQ:面向 FAQ 集合的问答系统框架、文本语义匹配工具 SimNet – Kaggle: Quora Question Pairs Text Classification • 2017-09 – Joulin et al. - 2016 - Bag of tricks for efficient text classification [pdf] [pdf (annotated)] [note] • 2017-10 – Kim - 2014 - Convolutional neural networks for sentence classification [pdf] [pdf (annotated)] [note] – Zhang and Wallace - 2015 - A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification [pdf] [pdf (annotated)] [note] – Zhang et al. - 2015 - Character-level convolutional networks for text classification [pdf] [pdf (annotated)] [note] – Lai et al. - 2015 - Recurrent Convolutional Neural Networks for Text Classification [pdf] [pdf (annotated)] [note] – Yang et al. - 2016 - Hierarchical attention networks for document classification [pdf] • 2017-11 – Iyyer et al. - 2015 - Deep unordered composition rivals syntactic methods for Text Classification [pdf] [pdf (annotated)] [note] • 2019-04 (Aspect level sentiment classification) – Wang et al. - 2016 - Attention-based LSTM for aspect-level sentiment classification [pdf] – Tang et al. - 2016 - Aspect level sentiment classification with deep memory network [pdf]

　– Chen et al. - 2017 - Recurrent Attention Network on Memory for Aspect Sentiment Analysis [pdf] – Xue and Li - 2018 - Aspect Based Sentiment Analysis with Gated Convolutional Networks [pdf] • references fastText text_classification – PyTorchText(知乎看山杯) Materials • Neural Networks for NLP (CS11-747 Spring 2019 @ CMU) • optimization algorithms – An overview of gradient descent optimization algorithms • NLP-progress • Awesome-Chinese-NLP • StateOfTheArt.ai • funNLP(从文本中抽取结构化信息等资源汇总) • StanfordNLP: Official Stanford NLP Python Library for Many Human Languages • Browse state-of-the-art