В. П. Захаров Корпусная лингвистика



бет3/3
Дата04.07.2016
өлшемі486.5 Kb.
#177258
түріУчебно-методическое пособие
1   2   3

Жанр текста


нежанровая проза
автобиографическая проза
детектив
детская литература
историческая проза
криминальная литература
приключения
фантастика
юмор и сатира


Рис. 5. Запросная форма НКРЯ для поиска по жанру.

Тип текста


автобиографическая проза
ассоциативная проза
очерк
письмо литературное
повесть
пьеса
рассказ
роман
сказка
эссе



Рис. 6. Запросные формы НКРЯ для поиска по типу текста.

Автор текста


Фамилия, имя


Пол: мужской женский любой

Год рождения: от до



Рис. 7. Запросная форма НКРЯ для поиска по автору.
Приложение 3

Фрагмент словаря-тезауруса по корпусной лингвистике

В структуре словарных статей выделяются поля, которые помечены следующими метками: Term\ англоязычный термин; Trans\ русскоязычный термин; Def\ определение; Syn\ синоним; Ant\ антоним; Up\ вышестоящий термин; Down\ нижестоящий термин, Cyt\ цитата.


Term\ aligned parallel corpus

Trans\ выровненный параллельный корпус

Def\ Parallel corpus where texts in one language and their translations into other languages are aligned, sentence by sentence, phrase by phrase.

Up\ parallel corpus

Cyt\ A type of multilingual corpus where texts in one language and their translations into other languages are aligned, sentence by sentence, preferably phrase by phrase. Sometimes reciprocate parallel corpora are set up, corpora containing authentic texts as well as translations in each of the languages involved. This allows double-checking translation equivalents.

Cyt\ A parallel corpus is not immediately user-friendly. For the corpus to be useful it is necessary to identify which sentences in the sub-corpora are translations of each other, and which words are translations of each other. A corpus which shows these identifications is known as an aligned corpus as it makes an explicit link between the elements which are mutual translations of each other. For example, in a corpus the sentences "Das Buch ist auf dem Tisch" and "The book is on the table" might be aligned to one another. At a further level, specific words might be aligned, e.g. "Das" with "The". This is not always a simple process, however, as often one word in one language might be equal to two words in another language, e.g. the German word "raucht" would be equivalent to "is smoking" in English.

Term\ aligned reciprocate parallel corpus

Trans\ выровненный двусторонний параллельный корпус

Def\ Reciprocate parallel corpus where texts and their translations are aligned, sentence by sentence, phrase by phrase.

Up\ reciprocate parallel corpus

Cyt\ A type of multilingual corpus where texts in one language and their translations into other languages are aligned, sentence by sentence, preferably phrase by phrase. Sometimes reciprocate parallel corpora are set up, corpora containing authentic texts as well as translations in each of the languages involved. This allows double-checking translation equivalents.

Cyt\ A parallel corpus is not immediately user-friendly. For the corpus to be useful it is necessary to identify which sentences in the sub-corpora are translations of each other, and which words are translations of each other. A corpus which shows these identifications is known as an aligned corpus as it makes an explicit link between the elements which are mutual translations of each other. For example, in a corpus the sentences "Das Buch ist auf dem Tisch" and "The book is on the table" might be aligned to one another. At a further level, specific words might be aligned, e.g. "Das" with "The". This is not always a simple process, however, as often one word in one language might be equal to two words in another language, e.g. the German word "raucht" would be equivalent to "is smoking" in English.

Term\ annotated corpus

Trans\ размеченный корпус

Def\ Corpus enhanced with additional linguistic information.

Syn\ tagged corpus

Ant\ unannotated corpus

Up\ corpus

Down\ phonetically transcribed corpus

Down\ parsed corpus

Cyt\ A type of corpus enhanced with various types of linguistic information (or tagged corpus). An annotated corpus may be considered to be a repository of linguistic information, because the information which was implicit in the plain text has been made explicit through concrete annotation.

Cyt\ More difficult is the question of annotated corpora. It is proposed that this term is used for any corpus which includes codes that record extra information -- provenance, analytical marks, etc. Again the annotations should be separable from the plain text in a simple and agreed fashion. A set of conventions for removing, restoring and manipulating annotations is necessary, especially as the next few years will see a large growth in the provision of annotated corpora. It is naive to expect that big corpora will remain easy to manage if they are full of various annotations; retrieval times are already critical.

Cyt\ For example, the form "gives" contains the implicit part-of-speech information "third person singular present tense verb" but it is only retrieved in normal reading by recourse to our pre-existing knowledge of the grammar of English. However, in an annotated corpus the form "gives" might appear as "gives_VVZ", with the code VVZ indicating that it is a third person singular present tense (Z) form of a lexical verb (VV). Such annotation makes it quicker and easier to retrieve and analyse information about the language contained in the corpus.

Cyt\ Enriched data: Many corpora have already been enriched with additional linguistic information such as part-of-speech annotation, parsing and prosodic transcription. Hence data retrieval from annotated corpora can be easier and more specific than with unannotated data.

Term\ balanced corpus

Trans\ исчерпывающий корпус

Look\ saturated corpus

Cyt\ A type of corpus composed according to parameters such as text type, genre or domain.

Cyt\ The Core Corpus of 2 million words is intended to be a representative subset of the whole corpus, in the sense that it contains samples from all the major subdivisions of the whole BNC, and in approximately the same proportions as those found in the BNC as a whole. There is one major exception to this statement however: whereas in the whole BNC, only c.10 million words (10% of the corpus) consist of spoken data, the Core Corpus is divided approximately equally between written and spoken material (c.1 million words each). It will generally be felt that in an ideal balanced corpus of the language, at least half of the material should be spoken English. It was only the impracticality of collecting and transcribing 50 million words of the spoken language which led to abandonment of this goal of "ideal balance" in the case of the whole BNC.

Term\ corpus

Trans\ корпус

Def\ Body of texts.

Down\ opportunistic corpus

Down\ saturated corpus

Down\ balanced corpus

Down\ Reference corpus

Down\ annotated corpus

Down\ tagged corpus

Down\ unannotated corpus

Down\ raw corpus

Down\ parsed corpus

Down\ treebank

Down\ phonetically transcribed corpus

Down\ monolingual corpus

Down\ multilingual corpus

Down\ monitor corpus

Down\ finite corpus

Down\ samples corpus

Down\ whole text corpus

Down\ whole text corpus

Down\ synchronic corpus

Down\ diachronic corpus

Cyt\ Corpora are sources of quantitative information beyond compare.

Cyt\ Leech (1992) argues that the corpus is a more powerful methodology from the point of view of the scientific method, as it is open to objective verification of results.

Cyt\ Whatever philosophical advantages we may eventually see in a corpus, it is the computer which allows us to exploit corpora on a large scale with speed and accuracy.

Cyt\ However, the notion of a corpus as the basis for a form of empirical linguistics is different from the examination of single texts in several fundamental ways.

Cyt\ In principle, any collection of more than one text can be called a corpus, (corpus being Latin for "body", hence a corpus is any body of text). But the term "corpus" when used in the context of modern linguistics tends most frequently to have more specific connotations than this simple definition.

Cyt\ We are therefore interested in creating a corpus which is maximally representative of the variety under examination, that is, which provides us with an as accurate a picture as possible of the tendencies of that variety, as well as their proportions.

Cyt\ Nowadays the term "corpus" nearly always implies the additional feature "machine-readable". This was not always the case as in the past the word "corpus" was only used in reference to printed text.

Cyt\ There is often a tacit understanding that a corpus constitutes a standard reference for the language variety that it represents. This presupposes that it will be widely available to other researchers, which is indeed the case with many corpora – e.g. the Brown Corpus, the LOB corpus and the London-Lund corpus.

Cyt\ Part-of-speech annotation is useful because it increases the specificity of data retrieval from corpora, and also forms an essential foundation for further forms of analysis (such as syntactic parsing and semantic field annotation).

Cyt\ Problem-oriented tagging (as described by de Haan (1984)) is the phenomenon whereby users will take a corpus, either already annotated, or unannotated, and add to it their own form of annotation, oriented particularly towards their own research goal.

Cyt\ In this session we will examine a few of the roles which corpora may play in the study of language. The importance of corpora to language study is aligned to the importance of empirical data. Empirical data enable the linguist to make objective statements, rather than those which are subjective, or based upon the individual's own internalised cognitive perception of language.

Cyt\ It is important to note that although many linguists may use the term "corpus" to refer to any collection of texts, when it is used here it refers to a body of text which is carefully sampled to be maximally representative of the language or language variety.

Cyt\ A linguist who has access to a corpus, or other (non-representative) collection of machine readable text can call up all the examples of a word or phrase from many millions of words of text in a few seconds. Dictionaries can be produced and revised much more quickly than before, thus providing up-to-date information about language. Also, definitions can be more complete and precise since a larger number of natural examples are examined.

Cyt\ Because a corpus is sampled to maximally represent the population, any findings taken from the corpus can be generalised to the larger population. Hence quantification in corpus linguistics is more meaningful than other forms of linguistic quantification because it can tell us about a variety of language, not just that which is being analysed.

Cyt\ Most European languages (not to mention Chinese, Japanese, Korean etc.) now have some sort of corpus already and there is a growing awareness that a good corpus can be put to many uses; hence their importance grows. Despite ini­tial disapprovals voiced by some linguists, doubts are dispelled by obvious and indisputable facts: nobody has ever been able to manually collect and subse­quently process so much data in his or her lifetime as the computer can in a very short time.

Cyt\ It may still be premature to try to mark out exhaustively what corpora may do for language studies and linguists; undoubtedly, many new options are still to come while the appetite of linguists is gradually whetted and new ways of corpus exploitation are offered by corpus linguists. In fact, it is hard to see a linguistic discipline not being able to profit from a corpus one way or another, both written and oral. It is increasingly clearer that new ways and methods for retrieving information from corpora will have to be given more thought.



Приложение 4

Миникорпус корпусной терминологии
(фрагмент)


Термин

Цитата

Corpus

Corpora are sources of quantitative information beyond compare.

Corpus

Leech (1992) argues that the corpus is a more powerful methodology from the point of view of the scientific method, as it is open to objective verification of results.

Corpus

Whatever philosophical advantages we may eventually see in a corpus, it is the computer which allows us to exploit corpora on a large scale with speed and accuracy.

Corpus

However, the notion of a corpus as the basis for a form of empirical linguistics is different from the examination of single texts in several fundamental ways.

Corpus

In principle, any collection of more than one text can be called a corpus, (corpus being Latin for "body", hence a corpus is any body of text). But the term "corpus" when used in the context of modern linguistics tends most frequently to have more specific connotations than this simple definition.

Corpus

We are therefore interested in creating a corpus which is maximally representative of the variety under examination, that is, which provides us with an as accurate a picture as possible of the tendencies of that variety, as well as their proportions.

Corpus

Nowadays the term "corpus" nearly always implies the additional feature "machine-readable". This was not always the case as in the past the word "corpus" was only used in reference to printed text.

Corpus

There is often a tacit understanding that a corpus constitutes a standard reference for the language variety that it represents. This presupposes that it will be widely available to other researchers, which is indeed the case with many corpora - e.g. the Brown Corpus, the LOB corpus and the London-Lund corpus.

Corpus

Part-of-speech annotation is useful because it increases the specificity of data retrieval from corpora, and also forms an essential foundation for further forms of analysis (such as syntactic parsing and semantic field annotation).

Corpus

Problem-oriented tagging (as described by de Haan (1984)) is the phenomenon whereby users will take a corpus, either already annotated, or unannotated, and add to it their own form of annotation, oriented particularly towards their own research goal.

Corpus

In this session we will examine a few of the roles which corpora may play in the study of language. The importance of corpora to language study is aligned to the importance of empirical data. Empirical data enable the linguist to make objective statements, rather than those which are subjective, or based upon the individual's own internalised cognitive perception of language.

Corpus

It is important to note that although many linguists may use the term "corpus" to refer to any collection of texts, when it is used here it refers to a body of text which is carefully sampled to be maximally representative of the language or language variety.

Corpus

A linguist who has access to a corpus, or other (non-representative) collection of machine readable text can call up all the examples of a word or phrase from many millions of words of text in a few seconds. Dictionaries can be produced and revised much more quickly than before, thus providing up-to-date information about language. Also, definitions can be more complete and precise since a larger number of natural examples are examined.

Corpus

Grammatical (or syntactic) studies have, along with lexical studies, been the most frequent types of research which have used corpora.

Corpus

Because a corpus is sampled to maximally represent the population, any findings taken from the corpus can be generalised to the larger population. Hence quantification in corpus linguistics is more meaningful than other forms of linguistic quantification because it can tell us about a variety of language, not just that which is being analysed.

Corpus

Most European languages (not to mention Chinese, Japanese, Korean etc.) now have some sort of corpus already and there is a growing awareness that a good corpus can be put to many uses; hence their importance grows. Despite ini­tial disapprovals voiced by some linguists, doubts are dispelled by obvious and indisputable facts: nobody has ever been able to manually collect and subse­quently process so much data in his or her lifetime as the computer can in a very short time.

Corpus

It may still be premature to try to mark out exhaustively what corpora may do for language studies and linguists; undoubtedly, many new options are still to come while the appetite of linguists is gradually whetted and new ways of corpus exploitation are offered by corpus linguists. In fact, it is hard to see a linguistic discipline not being able to profit from a corpus one way or another, both written and oral. It is increasingly clearer that new ways and methods for retrieving information from corpora will have to be given more thought.

Corpus

Since any language needs a consistent, perpetual and next-to-exhaustive coverage of its data, it should have a corpus of corresponding qualities, although in practice it is a gradual business of taking many minor decisions in the course of its construction and maintenance. This is particu­larly important in the case of small languages, which, unlike English and other languages, cannot afford the luxury of having a variety and multitude of corpora for specific purposes, at least not at the moment. What is really needed is a steady increase and perpetual growth of even, by present standards, very large corpora of billions of words, which should be as much representative as possible.

Corpus

Although the de­gree of the coverage of language by a large corpus is considerable, it is by no means true that today's corpora reflect language as a whole. Moreover, some corpus linguists are becoming more and more susceptible to another chal­lenge here, namely the degree of representativeness of this coverage, which is very much an open issue and matter of much dispute.

Corpus

As information is to be found coming from all fields of human life and activity, it is hard to imagine that corpora can be based on a collection of, per­haps, newspapers only. On the other hand, this diversity of sources suggests that a mapping of proportions in which various kinds of information occur should take place and be reflected in the design and structure of the corpus, should this be a general type of corpus. This raises the problem of the corpus representativeness, mentioned above.

Corpus

More generally, one may wonder where this trend actually fits in, in an attempt to pursue purely practical and utilitarian goals, or in one aiming at an exhaustive, systematic and non-eclectic description of one's language. Corpora definitely offer the latter possibility.

Corpus

Corpora are cross-sections of a discourse universe comprising all communication acts. The texts they monitor are principally transient communication acts.

Corpus

It is the task of the linguist to define and delimit the scope of the discourse universe she or he is interested in in such a way that it can be reduced to a corpus. Parameters can be language, time segment, region, situation, external and internal properties of texts, and many others.

Corpus collection

Corpus collection continued and diversified after the diary studies period: large sample studies covered the period roughly from 1927 to 1957 - analysis was gathered from a large number of children with the express aim of establishing norms of development.

Early corpus linguistics

All the work of early corpus linguistics was underpinned by two fundamental, yet flawed assumptions:

The sentences of a natural language are finite.



The sentences of a natural language can be collected and enumerated.

Содержание



1. Основные понятия ………………………………………………......

3

1.1. Введение: корпусы и корпусная лингвистика ……………...



1.2. Репрезентативность …………………………………………..

5

1.3. Размер корпуса ……………………………………………….



1.4. Разметка ……………………………………………….............

6

1.5. Технология создания корпусов ……………………………...

7

1.6. Автоматическая разметка ……………………………………

8

1.7. Исправление ошибок и снятие неоднозначности ………….

9

1.8. Форматы данных и стандартизация ………………………...



1.9. Корпусные менеджеры ………………………………............

10

1.10. Пользователи и способы использования корпусов …..............

11

1.11. Типы корпусов ………………………………………………

12

1.12. Терминология ……………………………………………….

14

2. Программа учебной дисциплины «Корпусная лингвистика» ......

15

2.1. Организационно-методический раздел ……………………..



2.2. Содержание курса ……………………………………............

16

2.3. Часть 1. Введение в корпусную лингвистику ….…………..



2.4. Часть 2. Создание корпусов ……….………………………...

20

2.5. Часть 3. Использование корпусов ………………….……….

25

Приложение 1. Корпусы в сети Интернет…………………………….

29

Приложение 2. Метаданные текстов в «Национальном корпусе
русского языка» (НКРЯ) ………………………………….............

35


Приложение 3. Фрагмент словаря-тезауруса по корпусной
лингвистике ………………………………………………………..

38


Приложение 4. Миникорпус корпусной терминологии ……..……...

44

Учебное издание

Виктор Павлович Захаров

КОРПУСНАЯ ЛИНГВИСТИКА
Учебно-методическое пособие
Зав. редакцией Г.И. Чередниченко

Редактор Н.Г. Михайлова

Технический редактор Л.Н. Иванова

Обложка А.В. Калининой

Подписано в печать с оригинала-макета 28.11.2005.
Ф-т 60х84/16. Усл. печ. л. 2,79. Уч.-изд. л. 1,83.

Тираж 120 экз. Заказ № .

РОПИ С.-Петербургского государственного университета.
199034, С.-Петербург, Университетская наб., 7/9.

Типография Издательства СПбГУ.


199061, С.-Петербург, Средний пр., 41.


1 Под словом здесь и далее имеется в виду словоупотребление (англ. token).



Достарыңызбен бөлісу:
1   2   3




©dereksiz.org 2024
әкімшілігінің қараңыз

    Басты бет