Facts and Figures about the Verbmobil Project

Facts and Figures about the Verbmobil Project

Reinhard Karger and Wolfgang Wahlster

DFKI GmbH, Saarbrücken, Germany

Abstract

In this chapter the organizational and funding structure of the Verbmobil project is summarized and the major technical data about the final Verbmobil system and the Verbmobil archives are compiled.

1. Verbmobil Milestones

The Verbmobil project was designed as an eight years initiative in speech and language technology, divided into two phases. The network plan for phase I was defined in 1992. Phase I started in January 1993. The first important milestone, the fully integrated Verbmobil Demonstrator, was reviewed by the international Verbmobil Advisory Board1 and technically and scientifically approved in February 1995 (16.2.95). The Verbmobil Demonstrator speaker independently recognized and processed spontaneously spoken German utterances within the context of appointment scheduling (1,292 word forms) and translated them into spoken English. It was unveiled to the public during CeBIT 1995 (13.3.95). The Verbmobil Research Prototype (2,500 word forms) was successfully reviewed (24.10.96) and presented at CeBIT 1997. It additionally recognized and processed Japanese utterances (400 word forms) and translated them into English.

Parallel to the research and development process the network plan for the second phase was defined in 1996 (February July). Phase II started in January 1997 and ends in September 2000. The Verbmobil Research Prototype IIa (German word forms: 8,323; English word forms: 4,484; Japanese word forms: 2,566) was reviewed in November 1998 (23.11.1998).

The Verbmobil system 1.0 works in three thematic domains: travel planning, appointment scheduling and remote PC maintenance. It is implemented as a stationary system and as a speech server for telephone applications. We will present the technical data of the Verbmobil system 1.0 below.

2. The Structure of the Verbmobil Project

2.1 BMBF Funding

The Verbmobil project was funded by the German Federal Ministry for Education and Research (BMBF) and additionally financed by the industrial partners. Until the year 2000, 116 million DM were allocated to the project. In addition the industrial partners brought in 53 million DM (see Table 1). Universities and research centers received a full 100% funding, while industrial partners contributed 60% of their costs.

Table 1. Funding

BMBF-Funding Phase I, 1.01.93 31.12.96

62,7 Mio. DM

BMBF-Funding Phase II, 1.01.97 - 30.9.2000

53,3 Mio. DM

Industrial investment I+II

32,6 Mio. DM

Related industrial R & D activities

ca. 20 Mio. DM

Total

168,6 Mio. DM

The project was controlled by the German Aerospace Research Establishment (DLR), Berlin.

2.2 The Management Structure of the Verbmobil Project

The Verbmobil project was jointly managed by the Scientific Management, the Verbmobil Steering Committee and the Group of Modul Managers (see Figure 1). The Scientific Management organised, coordinated and supervised the project and evaluated the progress of the implementation in close contact with the funding agency.

The assignment of the Verbmobil Steering Committee was to support the Scientific Management in ensuring the scientific excellence of the work. The Verbmobil Steering Committee had 19 meetings to adjust the project plan to new scientific results or trends in language technology and to react immediately to upcoming problems in the development and implementation process.

The Group of Module Managers was introduced in the beginning of phase II as an additional communication layer. The main task was to tie together and bundle the distributed development forces. In the 11 meetings of the Group of Module Managers all important decisions concerning interface definitions, offline data flow, delivery schedules, integration cycles were prepared, discussed and communicated. The implementation process was tightly monitored and supervised. It also provided contact points for the Scientific Management and gave feedback to the Verbmobil Steering Committee.

Fig. 1. The Management Structure of the Verbmobil Project

2.3 The Organizational Structure of the Verbmobil project

The Verbmobil project phase I was divided into 16 subprojects and 135 workpackages (see Table 2). 33 research groups from 28 universities, research centers and companies with a total of 125 persons per year were involved.

Table 2. The 16 subprojects of Verbmobil phase I

  1. Interaction of Signal Processing and Linguistics
  2. Integrated Speech Recognition Module
  3. Speech Recognition and Speaker Adaptation
  4. Speech Synthesis
  5. Lexicon and Morphology
  6. Syntax
  7. Formalism and Processing
  8. Semantic Construction
  9. Spontaneous and Incremental Generation
  10. Translation-oriented Dialog Processing
  11. Resolution for Translation
  12. Transfer
  13. Scenario and Interpreting Strategies
  14. Data Collection and Distribution of Spontaneous Speech Data
  15. Architecture
  16. Software Integration and Project Management
In phase II the project structure was revised and refocused. 8 subprojects with 127 workpackages were addressed by 24 research groups from 23 partners with 100 persons per year (see Table 3).

Table 3. The 8 subprojects of Verbmobil phase II

  1. Speech Recognition
  2. Linguistic Analysis
  3. Translation
  4. Dialog Processing and Context Resolution
  5. Generation and Synthesis
  6. Data Collection and Corpus Annotation
  7. System Design and Integration
  8. Project Management and Evaluation

To support the data flow needed for the distributed development a central FTP Server was installed in the first year of the project (5.11.93). Since then approximately 1 Terrabyte of data were transferred. The central Verbmobil Webserver (http://verbmobil.dfki.de) went online in 1995 (29.05.1995). It now consists of 600 pages with 4.5 MB of raw HTML code and 400 MB of data2.

3. The Verbmobil System

The Verbmobil system development was overall performed in 11 successive stepwise integration cycles. Each cycle lasted an average of 10 weeks. Version 1.0, which is now reached, consists of 69 modules communicating through 224 pools (including 26 pools for system configuration and control). The 224 pools used in the multi-blackboard architecture comprise 2,380 point-to-point connections. Modules are implemented in the programming languages: C, C++, LISP, Prolog, Fortran, Perl and Tcl/Tk. The fully fledged Verbmobil system is working with an average real-time factor of 3.4 5.3. (see Klüter et al., in this volume).

The final wordlists define the admissable words in word hypotheses graphs and the vocabulary that has to be processed. They actually consist of 19,594 word forms (German:10,157; English: 6,871; Japanese: 2,566). Naming expressions are excluded from this list and treated with class-based language models, so the system can actually process a larger vocabulary then specified by the wordlists (see Gibbon, Lüngen, in this volume).

To deal with the processing of repairs in spontaneous speech a module for self-corrections was implemented. In the Verbmobil corpus 21% of all turns contain at least one repair. The repair module was trained using a corpus of 12,558 turns (see Spilker et al., in this volume).

The semantic database, that is used as a central knowledge source for the deep analysis modules, consists of 15,385 entries (German: 7,874; English: 5,448; Japanese: 2,063) (see Kasper, in this volume). The module for semantic construction, SemCon, currently uses 91,762 Prolog predicates, 1,650 of which were hand written, the rest was compiled from lexical resources (see Schiehlen, in this volume). For the testing of the deep analysis, e.g. robust semantics, transfer and generation 46,951 VITs were used (German: 25,725; English: 21,226).

To realize the semantic transfer (see Emele et al., in this volume), the translation step in deep processing, a total of 22,782 expanded transfer rules are used (German <-> English: 18,088; German <-> Japanese: 4,694 expanded rules). The lexica used for generation consist of 18,522 stem entries (German: 8,032; English: 6,820; Japanese: 3,670) (see Becker et al., in this volume). The generation components for deep processing use a total of 13,640 microplanning rules (German: 7,170; English: 4,660; Japanese:1,810). The HPSG of Japanese has a hierarchy of 1,184 types. The HPSG of English has a hierarchy containing 2,015 types. The HPSG of German has a hierarchy containing 2,400 types.

The four shallow translation approaches use annotated dialogs or aligned language pairs for the offline training. Syndialog, the dialog-act based approach to translation, uses 334 multilingual finite state transducers (FST) for the analysis. They were derived from 30,000 utterances. For target language generation 149 sentence templates for each language and 30 multilingual generation FSTs are used (see Reithinger and Engel, in this volume).

StatTrans, a statistical approach to machine translation, was trained on a corpus of 58,332 (German/English) utterance pairs (running words German: 519,523; English: 549,921) (see Vogel et al., in this volume). 691,583 alignment templates (with maximal length 6) are used for generating the target language translation (German -> English: 368,539; English -> German: 323,044).

The Substring-based module was trained on a corpus of 24,680 German/English utterance pairs (running words German: 268,207; English: 292,367) (see Block, in this volume).

The Case-based translation module uses 30,000 templates for the bidirectional translation between German and English (see Auerswald, in this volume).

Verbmobil can be used hands-free. The system can be controlled with 46 speech commands (plus polite variants) in each language.

The Verbmobil system 1.0 was evaluated with a distributed large-scale web-based effort using GET (Graphical Evaluation Tool). 43,180 translations (German: 25,345; English: 17,835) were judged by 65 evaluators (sentence length 1 - 60 words) (see Tessiore, and v.Hahn, in this volume).

4. The Verbmobil Archives

In Verbmobil spontaneous speech data were collected and transcribed for the training and testing of Verbmobil systems. In this distributed effort 3,200 dialogs (German: 1,454; English: 726; Japanese: 1,020) uttered by 1,658 speakers (German: 1,022; English: 202; Japanese: 434) were recorded using a close microphone, a room microphone and a telephone (see Burger et al., in this volume). 79,562 turns were collected (German: 41,512; English: 16,104; Japanese: 21,946) with approximatively 1,520,000 running words (German: 670,000; English: 270,000; Japanese: 580,000). Overall 181,6 hours were recorded (German: 96.1; English: 37.9; Japanese: 47.7), transcribed and distributed on 56 CDs with a total of 21.5 GB. Analyzing the corpus 206,000 instances of articulatory background noise, 85,000 instances of breathing and 35,000 hesitations (voiced: 19,000, nasal: 2,500, vocalic-nasalized 13,500) were detected in the transcribed recordings. The Verbmobil data are distributed to research or commercial users via the Bavarian Archive of Speech Signals (BAS)3.

21 CDs with 1,505 German, English and Japanese dialogs were annotated with dialog acts (76,210 dialog acts using a hierachy of 32 dialog acts) (see Alexandersson et al., in this volume). The dialog act annotations were used to train the statistical dialog act classifier which is part of the dialog-act based approach to translation.

A treebank with 85,000 entries was developed (German: 35,000; English: 30,000; Japanese: 20,000). An annotated syntax tree provides information about: Part-of-Speech tags, phrasal categories, grammatical functions and root labels (Hinrichs et al. in this volume). The treebank was used for the training of the statistical parser, the chunk parser and the development of semantic construction rules and translation transfer rules.

4.1 Organisation, Documentation and Press Data

10 large project meetings4 and 3 milestone review meetings5 were organised in which the Verbmobil Adivisory Board and the BMBF/DLR were informed about the scientific progress and the actual Verbmobil system was presented6. Numerous workshops were organised to discuss the scientific matters, to fine tune details of the realization process, to synchronize the inter module communication.

During 7 years 238 Verbmobil Reports with overall 5,331 pages were written and published in the Verbmobil Report Series. They are available via the Verbmobil Webserver. Additionally 146 project internal Memos with overall 3,223 pages and 76 Technical Documents with 5,671 pages were written.

Verbmobil was presented at 5 press conferences7 and a variety of scientific or commercial occasions8. 232 press articles about Verbmobil were collected and documented in the Verbmobil Intranet9. Verbmobil was covered in a number of scientific TV programmes10.

5. The Verbmobil Consortium

In the Verbmobil Consortium a total of 31 different partners on three continents were involved (see Table 4). Verbmobil was a key resource for the education and training of researchers and engineers needed to build up language industry in Germany.

Table 4. Verbmobil Consortium (Partners only contributed to phase I are marked *)

Alcatel SEL AG, Stuttgart*
ATR International, Kyoto, Japan (Subcontractor)
CAP Debis Systemhaus GmbH, Fellbach*
Carnegie Mellon University, Pittsburgh, USA (Subcontractor)
Center for the Study of Language and Information, Stanford, USA
Christian-Albrechts-Universität Kiel*
DaimlerChrysler AG, Stuttgart
DASA / Temic, Ulm
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH, Kaiserslautern
Eberhard-Karls-Universität Tübingen
Friedrich-Alexander-Universität Erlangen-Nürnberg
Humboldt-Universität zu Berlin*
IBM Deutschland Informationssysteme GmbH, Stuttgart*
Institut der Gesellschaft zur Förderung der angewandten Informationsforschung e.V. an der Universität des Saarlandes, Saarbrücken*
Ludwig-Maximilians-Universität München
Philips GmbH, Forschungslaboratorien, Aachen
Ruhr-Universität Bochum
RWTH Aachen
Siemens Aktiengesellschaft, Berlin und München
Technische Universität Berlin
Technische Universität Braunschweig
Technische Universität Dresden
Technische Universität München
Universität Bielefeld
Universität Bonn
Universität des Saarlandes, Saarbrücken
Universität Hamburg
Universität Hildesheim*
Universität Karlsruhe
Universität Stuttgart
Universität Ulm*

During the duration of the Verbmobil project 369 scientists worked on Verbmobil positions. In addition a total of 919 researchers, including Master Students (238), Student Research Assistants (483), Internships (18), PhD Students (164) were involved.

 

References

Alexandersson, Engel, R., Kipp, M., Koch, S., Küssner, U., Reithinger, N., and Stede, M. Modeling Negotiation Dialogs. In this volume.

Auerswald, M. Example-Based Machine Translation with Translation Templates. In this volume.

Becker, T., Kilger, A., Lopez, P., and Poller,P. The Verbmobil Generation Component VM-GECO. In this volume.

Block, H., U. Example-based Incremental Synchronous Interpretation. In this volume.

Burger, S., Weilhammer, K., Tillmann H., and Schiel, F. Verbmobil Data Collection and Annotation. In this volume.

Emele, M., Dorna, M., Lüdeling, A., Zinsmeister, H., and Rohrer C. Semantic-Based Transfer. In this volume.

Gibbon, D., and Lüngen, H. Speech Lexica and Multilingual Vocabularies. In this volume.

Hinrichs, E.W., Bartels, J., Kawata Y., Kordoni, V., and Telljohann, H. The Tübingen Treebanks for Spoken German, English, and Japanese. In this volume.

Kasper, W., Multilingual Semantic Databases. In this volume.

Klüter, A., Ndiaye, A., and Kirchmann, H. Verbmobil from a Software Engineering Point of View: System Design and Software Integration. In this volume.

Reithinger, N., and Engel, R. Robust Content Extraction for Translation and Dialog Processing. In this volume.

Schiehlen, M. Semantic Construction. In this volume.

Spilker J., Klarner, M., and Görz, G. Processing Self-Corrections in a Speech-to-Speech System. In this volume.

Tessiore, L., and v. Hahn, W. Functional Validation of a Machine Interpretation System: Verbmobil. In this volume.

Vogel, S., Och, F.J., Tillmann, C., Niessen, S., Sawaf, H., and Ney, H. Statistical Methods for Machine Translation. In this volume.


1 We like to thank the members of the Verbmobil Advisory Board for the fruitful discussions: Prof. Dr. Susan Armstrong, Prof. Dr. Harry Bunt, Dr. Stephan Euler, Prof. Dr. Dieter Huber, Prof. Dr. Martin Kay, Dr. Ronald Kay, Dr. PH Herbert Reininger, Prof. Dr. Dieter Wolf

2 Webhits for the central Webserver: appr. 500,000 from 17,704 different domains (63% Germany, 13% US, 9% Europe, 4% Asia and from Canada, Australia, Guyana, Nicaragua, Costa Rica, Oman, United Arab Emirates, Bahrain, Iceland, Jamaica, South Africa)

3 More Information at: http://www.bas.uni-muenchen.de/bas

4 Project meetings: 27.-28.01.94, Heidelberg; 24.-26.10.94, München; 25.-27.10.95, Stuttgart; 20.-22.03.96, Aachen; 28.-29.04.97, Bonn: 5. Projektlenkungssitzung; 27.-28.10.97, Kaiserslautern; 11.-12.05.98, Bonn; 07.-08.12.98, Aachen; 11.-12.05.99, Stuttgart; 9.-10.12.99, München

5 Project review meetings: 16.-17.02.95, Kaiserslautern, 24.-25.10.96, München, 23.11.98, Kaiserslautern

6 During the large project meetings the ongoing work was presented in 129 talks. The 2,518 transparencies were documented and distributed on 1,733 pages.

7 Press conferences: CeBIT, Hannover 13.03.95; DaimlerChrysler, Stuttgart, 27.10.95; Philips, Aachen, 22.03.96; Siemens, München, 25.10.96; Philips, Aachen, 09.12.98

8 Selected Verbmobil presentations: CeBIT 95, Hannover, 8.-15.03.95; International Consumer Fair, Berlin, 26.08.-3.09.95; CeBIT 97, Hannover, 11.-19.3.97; ICASSP 97, München, 21.-24.4.97; ACL 97, Madrid, 07.07. - 12.07.97; International Consumer Fair,, Berlin, 30.08. - 07.09.97; VDE-Kongress, Stuttgart, 22.10.98; Paderborner Podium, Paderborn, 05.10.99; CeBIT 2000, Hannover, 24.02.00-01.03.00

9 Selected articles about Verbmobil: Frankfurter Rundschau, 04.01.1992; Neue Ruhr Zeitung, 18.01.1992; Süddeutsche Zeitung, 12.09.1992; Markt & Technik, 12.02.1993; Frankfurter Allgemeine Zeitung, 11.09.1993; Bild der Wissenschaft, Dez 93; Spectrum der Wissenschaft, Mrz 94; Focus, 06.06.1994; VDI - Nachrichten, 21.10.1994; Computerwoche, 13.01.1995; Süddeutsche Zeitung, 16.02.1995; Hannoversche Allgemeine Zeitung, 14.03.1995; Saarbrücker Zeitung, 14.03.1995; Handelsblatt, 14.03.1995; Frankfurter Allgemeine Zeitung, 15.03.1995; Computer Zeitung, 20.04.1995; Computerwoche, 21.04.1995; Hamburger Abendblatt, 25.06.1995; Stuttgarter Nachrichten, 28.10.1995; Associated Press, 28.10.1995; Süddeutsche Zeitung, 30.10.1995; Die Zeit, 03.11.1995; Frankfurter Allgemeine Zeitung, 11.11.1995; Industrieanzeiger, 20.11.1995; Wallstreet Journal Europe, 20.11.1995; Computerzeitung, 23.11.1995; VDI-Nachrichten, 24.11.1995; Westdeutsche Allgemeine Zeitung, 23.03.1996; Welt am Sonntag, 24.03.1996; Frankfurter Allgemeine Zeitung, 12.04.1996; Bonner Generalanzeiger, 03.08.1996; Computerzeitung, 12.09.1996; Funkschau, 13.09.1996; Blick durch die Wirtschaft, 13.09.1996; The daily Telegraph, 16.10.1996; Frankfurter Rundschau, 26.10.1996; Bild Zeitung, 26.10.1996; Financial Times, 30.10.1996; Hamburger Abendblatt, 24.11.1996; Focus, 07.01.1997; Spiegel, 27.01.1997; Spiegel Special, Mrz 97; Frankfurter Rundschau, 15.03.1997; Spektrum der Wissenschaft, Apr 97; Handelsblatt, 13.05.1997; Wirtschafts Woche, 03.07.1997; Bild der Wissenschaft, Aug 97; Financial Times, 10.09.1998; Manager magazin, Nov 98; Computer Bild, 04. Jan 99; Die Woche, 08. Jan 99; Computer Zeitung, 28. Jan 99; Tomorrow, Feb 99; Saarbrücker Zeitung, 12. Jan 00; Rheinischer Merkur, 13. Jan 00; Die Woche, 28. Jan 00; Wired, Mai 00; Handelsblatt, 23.6.2000

10 Selected TV programmes that cover Verbmobil: BBC, 27.04.95; SR, 08.06.95; Südwest 3, 12.11.95; Arte, 08.04.97; Spiegel-TV, 11.05.97; HR 3, 18.05.97; 3Sat, 16.6.00