The annual Germeval natural language processing event solicits German-language "shared tasks"; one of this year's proposed tasks from the University of Hamburg is Prediction of Intellectual Ability and Personality Traits from Text, which proposes to mine test subjects' essays as a predictor of IQ.
The University of Washington computational linguist Emily M Bender does a good job explaining how this is all kinds of wrong, from supporting the racially biased junk science of IQ testing to the ethical implications of mining subjects' text to predict "intelligence."
Bender also points out that one of the people before the shared task, David Scheffer, owns a "neuromarketing" consultancy that offers "automated personality analysis" and "automated personnel selection."
A reply in the thread invokes Frank Pasquale's "Second Wave of Algorithmic Accountability," and Pasquale weighs in: "so much depends on where the corpora come from, what was people's intent when they wrote/spoke, what contestable cultural assumptions are built into assessments of complexity from text."
The proposers of the shared task have posted a reply that is broadly dismissive of the ethical critique of their work, implicitly refusing to contemplate the possibility that it is unethical to undertake the kind of analysis they're doing for the purposes they're interested in, and instead are pleading with their critics for "dialog" about how it could be made ethical. This puts is pretty squarely in Pasquale's "Second Wave" discourse: asking whether something should be done, not how.
During an aptitude test, participants are asked to write freely associated texts to provided questions and images. Trained psychologists can predict behavior, long-term development, and subsequent success from those expressions. Paired with an IQ test and provided high school grades, prediction of intellectual ability from a text can be investigated. Such an approach would extend sole text classification and could reveal insightful psychological traits.
Operant motives are unconscious intrinsic desires that can be measured by implicit or operant methods, such as the Operant Motive Test (OMT) or the Motive Index (MIX) employs. During the OMT and MIX, participants are asked to write freely associated texts to provided questions and images. Trained psychologists label these textual answers with one of five motives and corresponding levels. The identified motives allow psychologists to predict behavior, longterm development, and subsequent success. For our task, we provide extensive amounts of textual data from both, the OMT and MIX, paired with IQ and high school grades (MIX) and labels (OMT).
With this task we aim to foster research within this context. This task is focusing on classifying German psychological text data for predicting the IQ and high school grades of college applicants as well as performing speaker identification by same image descriptions.
(Image: Cryteria, CC BY< /a>, modified)
