The State and Fate of Linguistic Diversity in the NLP World

Pratik Joshi*, Sebastin Santy*, Amar Budhiraja*
Kalika Bali, Monojit Choudhury

Language technologies contribute to promoting multilingualism and linguistic diversity around the world. However, only a very small number of the over 7000 languages of the world are represented in the rapidly evolving language technologies and applications. In this paper we look at the relation between the types of languages, resources, and their representation in NLP conferences to understand the trajectory that different languages have followed over time. Our quantitative investigation underlines the disparity between languages, especially in terms of their resources, and calls into question the "language agnostic" status of current models and systems. Through this paper, we attempt to convince the ACL community to prioritise the resolution of the predicaments highlighted here, so that no language is left behind.

Citation (BibTeX)

@inproceedings{joshi-etal-2020-state,
    title="The State and Fate of Linguistic Diversity 
    and Inclusion in the {NLP} World",
    author="Joshi, Pratik  and Santy, Sebastin and Budhiraja,
      Amar  and Bali, Kalika  and Choudhury, Monojit",
    booktitle="Proceedings of the 58th Annual Meeting 
    of the Association for Computational Linguistics",
    year="2020",
    doi="10.18653/v1/2020.acl-main.560",
}

Paper

  PDF
  Abstract

Dataset and Resources

We create a consistent data model to complement the existing ACL Anthology Corpus with data from later years and of non-ACL conferences. We do this by augmenting the corpus using Semantic Scholar’s API and scraping ACL Anthology itself. This is a consolidated dataset for 11 conferences with different attributes. Stay tuned :)

  Language Taxonomy   SPIKE Interface

Coverage

Interactive Plots

We plot T-SNE of the Language-Conference-Authors entity embeddings. As they are in the same space, they help us in understanding the relations which exist between them.

Embeddings Plot

Anim pariatur cliche reprehenderit, enim eiusmod high life accusamus terry richardson ad squid. 3 wolf moon officia aute, non cupidatat skateboard dolor brunch. Food truck quinoa nesciunt laborum eiusmod. Brunch 3 wolf moon tempor, sunt aliqua put a bird on it squid single-origin coffee nulla assumenda shoreditch et. Nihil anim keffiyeh helvetica, craft beer labore wes anderson cred nesciunt sapiente ea proident. Ad vegan excepteur butcher vice lomo. Leggings occaecat craft beer farm-to-table, raw denim aesthetic synth nesciunt you probably haven't heard of them accusamus labore sustainable VHS.

Anim pariatur cliche reprehenderit, enim eiusmod high life accusamus terry richardson ad squid. 3 wolf moon officia aute, non cupidatat skateboard dolor brunch. Food truck quinoa nesciunt laborum eiusmod. Brunch 3 wolf moon tempor, sunt aliqua put a bird on it squid single-origin coffee nulla assumenda shoreditch et. Nihil anim keffiyeh helvetica, craft beer labore wes anderson cred nesciunt sapiente ea proident. Ad vegan excepteur butcher vice lomo. Leggings occaecat craft beer farm-to-table, raw denim aesthetic synth nesciunt you probably haven't heard of them accusamus labore sustainable VHS.

Anim pariatur cliche reprehenderit, enim eiusmod high life accusamus terry richardson ad squid. 3 wolf moon officia aute, non cupidatat skateboard dolor brunch. Food truck quinoa nesciunt laborum eiusmod. Brunch 3 wolf moon tempor, sunt aliqua put a bird on it squid single-origin coffee nulla assumenda shoreditch et. Nihil anim keffiyeh helvetica, craft beer labore wes anderson cred nesciunt sapiente ea proident. Ad vegan excepteur butcher vice lomo. Leggings occaecat craft beer farm-to-table, raw denim aesthetic synth nesciunt you probably haven't heard of them accusamus labore sustainable VHS.
Microsoft Research, Bangalore, India