On auto-coding

An unlikely series confusions and mismaps in my early career resulted in my brief involvement with the forced alignment-industrial complex. I’m grateful to people like the excellent Michael Wagner who supported my work on this topic, but I’m glad to see other people who have a deeper interest in acoustic phonetics methodology (like the also-excellent Michael McAuliffe) take over the enterprise. Phonetics has not been a major research interest for me for some time now.

I published a single two-page paper (Gorman et al. 2011) on forced alignment in 2011, and somehow, it’s my most cited work. Perhaps for that reason, I receive a lot of requests to review work involving forced aligners. Two frustrating “tricks” are extremely common in this literature.

The first involves manipulating the phonetic dictionary as a way to auto-code (socio)phonetic variables; I’ll refer to this method as dictionary hacking. For instance, Yuan & Liberman (2011) studied American English ‘g-dropping’ using this method. For each word in ending in ing [ɪŋ] they add a competing pronunciation variant [ɪn]. As a result, the final phonemic alignment contains information about whether the overall model took each ing rendition to be [ɪŋ] or [ɪn]. This sort of works (neat!), but I don’t think it’s a particularly good way to auto-code. First, good HMM accoustic models represent phones (or diphones, or triphones) using mixtures of multi-variant Gaussians (GMMs), and such models are capable of representing phonetically disparate renditions as instances of the same mixture; they don’t really reflect linguists’ intuitions about allophony. Secondly, and specifically to the Yuan & Liberman approach, they ignore a third possibility for this variable: [in], a tense high front vowel with an apical nasal. In my “style a” (i.e., low attention paid to my speech), this variant alternates with lax [ɪn]; I rarely produce [ɪŋ]. Dealing with differnet variants is hard using the dictionary-hacking method. There is of course a simple solution here. You use the forced aligner as is to find reasonably good timestamps of the relevant intervals, you extract acoustic features from those intervals, and you feed them into a discriminative supervised machine learning system trained on a small amount of labeled data. (In some cases, relevant corpora already have sufficiently detailed phonetic transcriptions so no additional labeling is necessary.) Done right, this will produce strictly better (more human-annotator-like) results than dictionary hacking: discriminative models optimized specifically for the coding task at hand, provided with appropriate acoustic features, will be more accurate than the forced aligner’s generative HMM-GMM system optimized for an objective only distantly related to the question at hand.

The second trick involves using (mono-, di-, or tri-)phone GMMs from different dialects or languages to auto-code. I’ll refer to this as phone hacking. For example, if one has a Montreal French acoustic model and an American English acoustic model, one can use the forced aligner to determine whether a rendition of Scottish English r is are more like the Montreal French or American English r. Milne (2011, 2014), for exapmle, describes some early work of this type. Once again, this sort of works (jeepers!) but it has all the same sorts of problems, problems which could be fixed by once again using the forced aligner for approximate timing information (it’s reasonably good at that), extracting phonetic features from the relevant intervals, and then feeding them into discriminative models optimized to code whatever variants of r you’re interested in. There’s no excuse, really for using Montreal French acoustic models on your Scottish data.

In my opinion, dictionary hacking and phone hacking are unnecessarily lazy, sloppy solutions to coding problems that aren’t really all that hard in the first place, and I tell the editors as much when asked to review papers using these techniques. The discriminative approach is not only relatively easy for a computationally sophisticated phonetician, but was almost as easy a full two decades ago. Since I don’t really work in this area anymore, I don’t know if there’s a library for discriminative auto-coding as well-designed or well-documented as the Montreal Forced Aligner, but if not, something like this is greatly needed.

References

Gorman, K., Howell, J. and Wagner, M. 2011. Prosodylab-Aligner: A tool for forced alignment of laboratory speech. Journal of the Canadian Acoustical Association 39(3): 192-193.
Milne, P. 2011. The effects of syllable position on allophonic variation in Québec French /ʀ/: A corpus analysis using a modified version of the Penn Phonetics Lab Forced Aligner. Paper presented at NWAV 40.
Milne, P. 2014. The variable pronunciations of word-final consonant clusters in a force aligned corpus of spoken French. Doctoral dissertation, University of Ottawa.
Yuan, J., and Liberman, M. 2011. Automatic detection of “g-dropping” in American English using forced alignment. In 2011 IEEE workshop on automatic speech recognition & understanding, pages 490-493.

More Pynchonian eye dialect

Twelve years ago I wrote a bit about Pynchon’s use of eye dialect in his underappreciated 2013 novel Bleeding Edge. In that book, the dialogue of Californian woman (Vyrna McElmo) is stylized so that her -ings are spelled -een, presumably denoting [in]; e.g., “I’m still, like, vibrateen“. I am now working through Vineland (1990). In that book, another Californian, DEA agent Hector Zuñiga, uses a different eye dialect take on the same variable: his -ings are spelled -ín, presumably denoting something similar, as in the following passage (p. 28):

All of you are still children inside, livín your real life back then. Still waitín for that magic payoff. […] Rill puzzlín.

I wonder if there’s a prosodic difference between (Caucasian) McElmo and (Latino) Zuñiga’s renditions of -ing in Pynchon’s mind, though.

“Indic” considered harmful

Indic is an adjective referring to the Indo-Aryan languages such as Hindi-Urdu or Bengali. These languages are spoken mostly in the northern parts of India, as well as in Bangladesh, Pakistan, Sri Lanka, Nepal, and the Maldives. This term can be confusing, because hundreds of millions of people in the Indian subcontinent (and nearby island nations) speak non-Indic first languages: over 250 million people, particularly in the south of India and the north of Sri Lanka, speak Dravidian languages, which include Malayalam, Tamil, and Telugu. Austronesian, Tibeto-Burman, and Tai-Kadai languages, and many language isolates, are also spoken in the India and the other nations of subcontinent, as is English (and French, and Portuguese). Unfortunately, there is now a trend to use Indic to mean ‘languages of the subcontinent’. See here for a prominent example. This is a new sense for Indic, and while there is probably a need for such a lexeme to express the notion (language of India or subcontinental language would work), reusing Indic, which already has a distinct and well-established sense, just adds unnecessary confusion.

A minor syntactic innovation in English: “BE crazy”

I recently became aware of an English syntactic construction I hadn’t noticed before. It involves the predicate BE crazy, which itself is nothing new, but here the subject of that predicate is, essentially, quoted speech from a second party. I myself am apparently a user of this variant. For example, a friend told me of someone who describes themselves (on an online dating platform) as someone who …likes travel and darts, and I responded, simply, Likes darts is crazy. That is to say, I am making some kind of assertion that the description “likes darts”, or perhaps the speech act of describing oneself as such, is itself a bit odd. Now in this case, the subject is simply the quotation (with the travel and part elided), and while this forms a constituent, a tensed VP, we don’t normally accept them as the subject of predicates. And I suspect constituenthood is not even required. So this is distinct from the ordinary use of BE crazy with a nominal subject.

I suspect, though I do not have the means to prove, this is a relatively recent innovation; I hear it from my peers (i.e., those of similar age, not my colleagues at work, who may be older) and students, but not often elsewhere. I also initially thought it might be associated with the Mid-Atlantic but I am no longer so sure.

Your thoughts are welcome.

Optionality as acquirendum

A lot of work deals with the question of acquiring “optional” or “variable” grammatical rules, and my impression is that different communities are mostly talking at cross-purposes. I discern at least three ways linguists conceive of optionality as something which the child must acquire.

Some linguists assume—I think without much evidence—that optionality is mere “free variation”, so that the learner simply needs to infer which rules bear a binary [optional] feature. This is an old idea, going back to at least Dell (1981); Rasin et al. (2021:35) explicitly state the problem in this form.
Variationist sociolinguists focus on the differential rates at which grammatical rules apply. They generally recognize the acquirenda as essentially conditional probability distributions which give the probability of rule application in a given grammatical context. Bill Labov is a clear avatar of this strain of thinking (e.g., Labov 1989). David Adger and colleagues have attempted to situate this within modern syntactic frameworks (e.g., Adger 2006).
Some linguists believe that optionality is not statable within a single grammar, and must reflect the competing grammars. The major proponent of this approach is Anthony Kroch (e.g., Kroch 1989). While this conception might license some degree of “nihilism” about optionality, it also has led to some interesting work which hypothesizes interesting substantive constraints on grammar-internal constraints on variation as in the work of Laurel MacKenzie and colleagues (e.g., MacKenzie 2019). This work is also very good at ridding the (2) of some of its unfortunate “externalist” thinking.

I have to reject (1) as overly simplicistic. I find (2) and (3) both compelling in some way but a lot of work remains to synthesize or adjudicate between them.

References

Adger, D. 2006. Combinatorial variability. Journal of Linguistics 42(3): 503-530.
Dell, F. 1981. On the learnability of optional phonological rules. Linguistic Inquiry 12(1): 31-37.
Kroch, A. 1989. Reflexes of grammar in patterns of language change. Language Variation & Change 1(1): 199-244.
Labov, W. 1989. The child as linguistic historian. Language Variation & Change 1(1): 85-97.
MacKenzie, L. 2019. Perturbing the community grammar: Individual differences and community-level constraints on sociolinguistic variation. Glossa 4(1): 28.
Rasin, E., Berger, I., Lan, R., Shefi, I., and Katzir, R. 2021. Approaching explanatory adequacy in phonology using Minimum Description Length. Journal of Language Modelling 9(1): 17-66.

The different functions of probabilty in probabilistic grammar

I have long been critical of naïve interpretations of probabilistic grammar. To me, it seems like the major motivation for this approach derives from a naïve—I’d say overly naïve—linking hypothesis mapping between acceptability judgments and grammaticality, as seen in Likert scale-style acceptability tasks. (See chapter 2 of my dissertation for a concrete argument against this.) But in this approach, the probabilities are measures of wellformedness.

It occurs to me that there are a number of ontologically distinct interpretations of grammatical probabilities of the sort produced by “maxent”, i.e., logistic regression models.

For instance, at M100 this weekend, I heard Bruce Hayes talk about another use of maximum entropy models: scansion. In poetic meters, there is variation in, say, whether the caesura is masculine (after a stressed syllable) or feminine (after an unstressed syllable), and the probabilities reflect that.¹ However, I don’t think it makes sense to equate this with grammaticality, since we are talking about variation in highly self-conscious linguistic artifacts here and there is no reason to think one style of caesura is more grammatical than the other.²

And of course there is a third interpretation, in which the probabilities are production probabilities, representing actual variation in production, within a speaker or across multiple speakers.

It is not obvious to me that these facts all ought to be modeled the same way, yet the maxent community seems comfortable assuming a single cognitive model to cover all three scenarios. To state the obvious, it makes no sense for a cognitive model to account for interspeaker variation because there is no such thing as “interspeaker cognition”, there are just individual mental grammars.

Endnotes

This is a fabricated example because Hayes and colleagues mostly study English meter—something I know nothing about—whereas I’m interested in Latin poetry. I imagine English poetry has caesurae too but I’ve given it no thought yet.
I am not trying to say that we can’t study grammar with poetry. Separately, I note, as did, I think, Paul Kiparsky at the talk, that this model also assumes that the input text the poet is trying to fit to the meter has no role to play in constraining what happens.

Myths about writing systems

In collaboration with Richard Sproat, I just published a short position paper on “myths about writing systems” in NLP to appear in the proceedings for CAWL, the ACL Workshop on Computation and Writing Systems. I think it will be most of all useful to reviewers and editors who need a resource to combat nonsense like Persian is a right-to-left language and want to suggest a correction. Take a look here.

Linguistics and prosociality

It is commonly said that linguistics as a discipline has enormous prosocial potential. What I actually suspect is that this potential is smaller than some linguists imagine. Linguistics is of course essential to the deep question of “what is human nature”, but we are up against our own epistemic bounds in answering these questions and the social impact of answering this question is not at all clear to me. Linguistics is also essential to the design of speech and language processing technologies (despite what you may have heard: don’t believe the hype), and while I find these technologies exciting, it remains to be seen whether they will be as societically transformative as investors think. And language documentation is transformative to some of society’s most marginalized. But I am generally skeptical of linguistics’ and linguists’ ability to combat societal biases more generally. While I don’t think any member of society should be considered well-educated until they’ve thought about the logical problems of language acquisition, considered the idea of language as something that exists in the mind rather than just in the ether, or confronted standard language ideologies, I have to question whether the broader discipline has been very effective here getting these messages out.

Noam and Bill are friends

One of the more confusing slanders against generativism is the belief that it has all somehow been undone by William Labov and the tradition of variationist sociolinguistics. I have bad news: Noam and Bill are friends. I saw them chopping it up once, in Philadelphia, and I have to assume they were making fun of functionalists. Bill has nice things to say about the generativist program in his classic paper on negative concord; Noam has some interesting comments about how the acquirenda probably involve multiple competing grammars in that Piaget lecture book. They both think functionalism is wildly overrated. And of course, the i-language perspective that Noam brings is an absolute essential to dialogues about language ideologies, language change, stigma and stratification, and so forth that we associate with Bill.

Neurolinguistic deprogramming

I venture to say most working linguists would reject—outright—strong versions of linguistic relativity and the Sapir-Whorf hypothesis, and would regard neuro-linguistic programming as pseudoscientific rubbish. This is of course in contrast to the general public: even the highly-educated take linguistic relativity as an obvious description of human life. Yet, it is not uncommon for the same linguists to endorse beliefs in the power of renaming that is hard to reconcile with the general disrepute of the vulgar Whorfian view the power of renaming assumes.

For instance, George Lakoff’s work on “framing” in politics argued that renaming social programs was the one weird trick needed to get Howard Dean into the White House. While this seems quaint in retrospect, his proposal was widely debated at the time. Pinker’s (sigh) takedown is necessary reading. The problem, of course, is that Lakoff ought to have provided, and ought to have been expected to provide, any evidence at all for a view of language widely regarded as untutored by his colleagues.

The case of renaming languages is a grayer one. I believe that one ought to call people what they want to be called, and that if stakeholders would prefer their language to be referred to as Tohono Oʼodham rather than Pápago, I am and will remain happy to oblige.¹ If African American Vernacular English is renamed to African American Language (as seems to be increasing common in scholarship), I will gladly follow suit. But I can’t imagine how it could be the case that the renaming represents a reconceptualization of either the language itself, or a change in how we study it. Indeed, it would be strange for the name of any language to reflect any interesting property of said language. French by any other name would still have V-to-T movement and liaison.

It may be that these acts of renaming have power. Indeed, I hope they do. But I have to suspect the opposite: they’re the sort of fiddling one does when one is out power, when one is struggling to believe that a better world is possible. And if I’m wrong, who is better suited to show that than the trained linguist?

Endnotes

Supposedly, the older name of the language comes from a pejorative used by a neighboring tribe, the Pima. Ba꞉bawĭkoʼa means, roughly ‘tepary bean eater’. The Spanish colonizers adapted this as Pápago. I feel like the gloss sounds like a cutting insult in English too, so I get why this exonym has fallen in disrepute.