Data Science for Linguists 2024

Course home for
     LING 1340/2340

HOME
• Policies
• Term project guidelines
• Learning resources by topic
• Schedule table

Homework 2: Process the ETS Corpus

This homework carries a total of 80 points.

NOTE: To give you all more time, I am staggering this homework submission into two stages.
Goal 1 and Goal 2 are due on 2/2 (Fri); the rest are due on 2/5 (Mon).
See the "Submission" section at the bottom for details.

For this homework, we will explore and process the ETS Corpus of Non-Native Written English, using Python’s numpy and pandas libraries we recently learned.

Dataset

Repo, files and folders

Goals

The first goal of this work is Basic Data Processing, which involves processing of CSV files and text files. You should build two DataFrame objects: essay_df and prompt_df.

Building essay_df (see screenshot):

  1. Start with index.csv and build a DataFramed named essay_df. Specifics:
    • Rename 'Language' column name to 'L1', which is a more specific terminology.
    • Likewise, change 'Score Level' to 'Score'. Single-word column names are more handy. (Why?)
  2. Augment the essay_df DataFrame with an additional column named 'Partition' with three appropriate values: ‘TS’ for testing, ‘TR’ for training, and ‘DV’ for development.
    • This information you can get from the three additional CSV files, which split the data into thee partitions: training, testing, and development sets.
  3. Create a new column called 'Text', which stores the string value of each essay content. You will need to find a way to read in each essay text stored as individual files.
  4. As you work on EDA and linguistic analysis, you may create additional columns to hold new data points. I’ll leave these up to you. Initially, the DataFrame should look like the screenshot above.

Building prompt_df (see screenshot):


The second goal is Exploratory Data Analysis (EDA).

  1. Read up on documentation in order to gain understanding of the data set. There is a README file, a PDF document, and the LDC publication page. What is the purpose of this data, what sort of information is included, and what is the organization?
  2. Then, explore the data to confirm the content. For example, the PDF document contains tables illustrating the make-up of the data and various data points. Don’t take their word for it! You should find a way to confirm and demonstrate these data points through your code.
  3. Visualization: Try out at least one plot/graph.

The third and last goal is Linguistic Analysis. In particular, we want to be able to highlight quantitative differences between three response groups: low, medium and high levels of writing proficiency. Explore the following:

  1. Text length: Do learners write longer or shorter responses?
    → Can be measured through average text length in number of words
  2. Syntactic complexity: Are the sentences simple and short, or are they complex and long?
    → Can be measured through average sentence length
  3. Lexical diversity: Are there more of the same words repeated throughout, or do the essays feature more diverse vocabulary?
    → Can be measured through type-token ratio. Find a way to address the big caveat.
  4. Vocabulary level: Do the essays use more of the common, everyday words, or do they use more sophisticated and technical words?
    → a. Can be measured through average word length (common words tend to be shorter), and
    → b. Can be measured against published lists of top most frequent English words.

There are five total measurements. Choose four if you are relatively new to programming; experienced programmers should work on all five. Importantly, you must employ statistical tests such as the t-test or one-way ANOVA to demonstrate whether or not a difference is significant.

Additional pointers on analysis: If you are going for 4b., which is the most involved kind of task, you might want to revisit this “Bulgarian vs. Japanese EFL Writing” homework from Intro to CL.

Your report, aka Jupyter Notebook file

Submission

  1. When you think you are finished, “Restart & Run All” your Jupyter notebook one last time, so the output is all orderly and tidy. Save the file.
  2. After the usual local Git routine, push to your own GitHub repo one last time. Check it to make sure your file is there and everything looks OK.
  3. That’s it! Since the repo is shared with me and the TAs, we have access.
  4. NEW: to give you more time, I'm having you submit in two stages.
    • 2/2 (Fri) Finish Goal 1 (“Basic data processing”) and Goal 2 (“EDA”).
    • 2/5 (Mon) Complete work, including Goal 3 (“Linguistic Analysis”).
    • No need to make separate JNB files for the two submissions! That’s what git versions are for.