Frederick Jelinek

Frederick Jelinek
Born	(1932-11-18) 18 November 1932 Kladno, Czechoslovakia
Died	14 September 2010(2010-09-14) (aged 77)
Occupation(s)	Information theorist, researcher in automatic speech recognition and natural language processing
Years active	1950s–2010
Known for	Contributions to information theory, speech recognition, natural language processing; "Every time I fire a linguist" remark
Notable work	"Continuous Speech Recognition by Statistical Methods" (1976), Statistical Methods for Speech Recognition (1997)

Frederick Jelinek (November 18, 1932 – September 14, 2010) was a notable Czech-American expert specializing in information theory, automatic speech recognition, and natural language processing. He is famously remembered for the remark, "Every time I fire a linguist, the performance of the speech recognizer goes up."

Born in Czechoslovakia prior to World War II, Jelinek emigrated to the United States during the onset of communist rule. He pursued engineering studies at the Massachusetts Institute of Technology and spent a decade teaching at Cornell University before joining IBM Research. In 1961, he married Milena Jelinek, a Czech screenwriter. At IBM, he led efforts that significantly advanced computer speech recognition and machine translation. Later, he directed the Center for Language and Speech Processing at Johns Hopkins University for 17 years until his passing.

Personal life

Bedřich Jelínek, later known as Frederick Jelinek, was born on November 18, 1932, in Kladno to parents Vilém and Trude Jelínek. His father was Jewish, while his mother was born in Switzerland to Czech Catholic lineage and converted to Judaism. Although Vilém intended to flee Nazi occupation, he remained and perished in the Theresienstadt concentration camp in 1945. Due to his mother's heritage, Jelinek and his family were spared from concentration camps but were forcibly relocated to Prague in 1941.

After the war, despite having missed several school years because of restrictions imposed on Jewish children, Jelinek attended gymnasium. His mother worked diligently to secure their emigration, especially as he encountered obstacles taking his graduation exams. While his mother wished for him to become a doctor, Jelinek himself aimed to study law. He initially studied engineering at the City College of New York and received financial support from the National Committee for a Free Europe, which enabled him to attend MIT where he completed a Ph.D. in 1962 under the mentorship of Robert Fano.

In 1957, during a visit to Prague, Jelinek met film student Milena Tobolová through Miloš Forman. Tobolová’s dissident background complicated their relationship; however, with advocacy from individuals such as Jerome Wiesner and Cyrus Eaton, she was able to emigrate following President Kennedy’s inauguration in 1961.

Following his graduate studies, Jelinek intended to collaborate with linguist Charles F. Hockett at Cornell but instead continued focusing on information theory. After a sabbatical at IBM beginning in 1972, he joined full-time in 1974. Initially brought on as a researcher, he soon became head of the Continuous Speech Recognition group at IBM’s Thomas J. Watson Research Center. Despite his important contributions, his work remained largely unknown within Czechoslovakia due to limitations on scientists attending international conferences.

After the collapse of communism in 1989, Jelinek actively promoted scientific cooperation with Czech institutions and contributed to founding an IBM computing center at Charles University. He retired from IBM in 1993 and accepted the role of director and Julian Sinclair Smith Professor of Electrical and Computer Engineering at Johns Hopkins University’s Center for Language and Speech Processing. He remained there until his death from a heart attack in September 2010. He is survived by his wife, two children, a sister, stepsister, and three grandchildren.

Research and legacy

During the mid-1950s, Information theory rose to prominence; however, pioneer Claude Shannon cautioned in 1956 that excessive reliance on this framework could lead to disappointment, as concepts such as information, entropy, and redundancy did not provide solutions to all challenges. By the 1960s, developments including Noam Chomsky's 1957 work Syntactic Structures, which critiqued probabilistic models for their lack of syntactic understanding, alongside the 1966 ALPAC report that recommended halting government funding for machine translation research, effectively impeded advancement in natural language processing (NLP).

Jelinek’s fascination with linguistics was sparked by his wife’s participation in MIT’s linguistics program. Although initially facing opposition from his adviser Fano, Jelinek resumed his research at IBM, where the research focus contrasted significantly with other groups. According to Mark Liberman, while Jelinek headed IBM’s efforts on the general dictation problem after 1972, most U.S. companies and academic institutions either concentrated on narrow problems or avoided the field entirely.

Jelinek tackled speech recognition through the lens of information theory—modeling it as a noisy channel conveyed by acoustic signals—a perspective considered innovative at that time. Their initial model introduced the notion of perplexity and was published in 1976 under the title "Continuous Speech Recognition by Statistical Methods" in Proceedings of the IEEE. This noisy channel framework reduced speech recognition to constructing two statistical models. Their subsequent system, Tangora, expanded upon this by employing n-gram models, especially trigrams. Although initially perceived as overly simplistic, this trigram approach maintained its influence until Jelinek’s 1999 publication. The same methodology applied to phonemes within single words also gave rise to tagging techniques now widely used in NLP applications.

IBM’s gradual research approach became predominant after DARPA restarted NLP research in the mid-1980s, establishing standardized methodologies with common objectives, datasets, and evaluation metrics. The Continuous Speech Recognition Group’s data-driven investigations contributed significantly to founding the Linguistic Data Consortium. While speech recognition remained an unsolved problem throughout the 1980s, IBM aimed to extend these approaches to machine translation and stock market forecasting; some researchers later transitioned to Renaissance Technologies. Jelinek observed that comparable techniques form the foundation of most contemporary machine translation systems. Commentators have noted that engineering innovations have propelled scientific advancements in this domain, reversing previous trends.

Jelinek received numerous "best paper" honors and corporate awards at IBM. His distinctions include the IEEE Signal Processing Society's Society Award (1997), the ESCA Medal for Scientific Achievement (1999), an IEEE Third Millennium Medal (2000), the Antonio Zampolli Prize from the European Language Resources Association (2004), the IEEE James L. Flanagan Speech and Audio Processing Award (2005), and the Association for Computational Linguistics Lifetime Achievement Award (2009). He was awarded an honoris causa Ph.D. by Charles University (2001), elected to the National Academy of Engineering (2006), and named one of twelve inaugural fellows of the International Speech Communication Association (2008).

"Each time a linguist is dismissed, the speech recognizer's performance improves"

Jelinek is widely remembered for the remark, "Each time a linguist is dismissed, the speech recognizer's performance improves," though the precise phrasing, timing, and circumstances vary. Daniel Jurafsky and James H. Martin recall Jelinek expressing in December 1988 that "Whenever a linguist leaves the team, the recognition accuracy increases," while Roger K. Moore referenced a 1985 workshop version: "Every time we let go of a phonetician or linguist, our system’s performance gets better."

Steve Young shared an account where, following the resignation of a linguist, Jelinek replaced them with an engineer, resulting in significant enhancements to the system; this scenario repeated after another linguist departed.

During a 2004 talk, Jelinek explained that he first made this statement at a 1988 workshop held in Wayne, Pennsylvania. He recounted how IBM’s initial Continuous Speech Recognition group included two linguists who were unable to produce effective grammars. Once they exited, the predominantly engineering and physics team—with strong expertise in information theory—attained superior outcomes.

Selected publications

• Jelinek, Frederick (1968). Probabilistic Information Theory: Discrete and memoryless models. McGraw-Hill. 689 pages.
• ———————- (1969). "Fast sequential decoding algorithm using a stack". IBM Journal of Research and Development 13(6):675–685.
• ———————- (1969). "Tree encoding of memoryless time-discrete sources with a fidelity criterion". IEEE Transactions on Information Theory 15(5):584–590. (Awarded Best Paper in 1971)
• Bahl, Lalit R.; John Cocke, Frederick Jelinek, Josef Raviv (1974). "Optimal decoding of linear codes for minimizing symbol error rate". IEEE Transactions on Information Theory 20(2):284–287.
• ———————- (1976). "Continuous speech recognition by statistical methods". Proceedings of the IEEE 64(4):532–556.
• Brown, P.; Cocke et al. (1988). "A statistical approach to language translation". In Dénes Vargha, ed. Coling 88. pp. 71–76.
• ———————- (1990). "Self-Organized Language Modeling for Speech Recognition". In Waibel & Lee, eds. Readings in speech recognition. Morgan Kaufmann.
• ———————-; John D. Lafferty and Robert L. Mercer (1990). "Basic methods of probabilistic context free grammars". IBM Technical Report RC 16374.
• ———————- (1997). Statistical Methods for Speech Recognition. MIT Press. 283 pages.
• Chelba, Ciprian; Frederick Jelinek (2000). "Structured Language Modeling". Computer Speech & Language 14(4):283–332. (Received Best Paper award in 2002)
• Xu, Peng; Ahmad Emami and Frederick Jelinek (2003). "Training Connectionist Models for the Structured Language Model". In Collins & Steedman, eds. EMNLP '03. pp. 160–167. (Best paper award)

References

Notes

References

External links

• Institutional page at Johns Hopkins University