Possible output of Colonial Valley Zapotec dictionary

Maybe it is because I have been working with output formats for Copala Triqui and San Dionisio Ocotepec Zapotec dictionaries lately, but I spent some time today thinking about what an eventual output format for a Colonial Valley Zapotec dictionary might look like.

This is one draft of a format to consider, where the entries have the form:

CVZ orthography  (part of speech) English gloss Spanish gloss (Examples) {Cordova page numbers, references to other sources such as Whitecotton} {Cordova original entry/entries} (alternate spellings)

Here is one entry fleshed out with examples, with several Cordova entries merged together …

I also put the different languages in different colors to make it easier to distinguish the different parts of the entry from each other.  Examples here are chosen to illustrate some of the range of different morphology that the root chono appears with.  I pulled them from the corpus using the 'Find Example' feature in FLEx and edited them for length.  I also italicized the word or phrase that contains chono and the corresponding part of the translation to make it easier for the reader.

(One thing I can see already is that I have been inconsistent in the abbreviations for the names of the texts -- sometimes Feria and sometimes Doctrina!)

A posted version of a dictionary like this would allow users who don't have access to the full FLEx project to nevertheless search through it and be able to advance over what is currently available to them.  (Of course, I think what we have still needs lots of clean up before we would have a version that would be ready for posting.)

Central Zapotec languages

The tree below is the current version of something that I am working on.  It's intended to show major branches of Central Zapotec (according to the classification of Smith Stark 2007), and it also shows (in brackets and italics) where we currently have Colonial Valley Zapotec documents in our FLEx corpus.

I know that there are many other Colonial Valley Zapotec documents out there, so the tree will get 'bushier' over time, but I found this a useful graphic view of where the documents come from in terms of distribution.

More examples of regular expressions in FLEx

This regular expression is used to help me separate out the right allomorph of the habitual prefix of the Colonial Valley Zapotec verb.  The habitual aspect comes in a few allomorphic variants, which are orthographically usually written <te, to, ti>.  I want to strip this from the citation form and put it in a separate field called Cordoba habitual form.

The procedure is to copy the whole citation form to the Cordova habitual form, then use a regular expression in bulk edit to remove everything except the prefix.   The following search and replace comes fairly close to what I want:

This bulk replace operation looks for the beginning of the record (^), then any number of characters (.*), then t followed by one character (\w).   I put the sequence t & one character inside the capturing parentheses, because I want to be able to refer to it in the Replace operation on the next line.

After the parenthesis, I have any number of characters (.*) and the end of the record ($).

This is replaced by t, whatever the letter that was captured in the previous parenthesis was ($1) and a hyphen.

So if the first line finds   blah blah ti+capaya blah blah  it will be replaced by ti-.

This mostly works -- except that the way the original Cordova entries came to me, there are sometime several Zapotec verbs listed together in a single entry; possibly with differential potential prefixes.  So I have to give the results a visual inspection to make sure nothing has gone wrong before hitting the Apply button in Bulk Edit.  If there is a record where this will give the wrong result, I can just uncheck it, and edit it manually.

The counter part to this regular expression is the one that takes the citation form, strips off the habitual prefix and returns just the portion minus that prefix.   The search and replace that will do that is the following:

The Find expression looks for a t followed by one word-forming character and a literal (+) at the beginning of a record (^).  It starts to capture everything from here to the end of the record ($) and stores it in the variable $1.   When this search and replace is applied, the effect is to take a word like

ti+capaya

and replace it with capaya.

More sophisticated regular expression searching in the Cordova

I am not a computational linguist, by any means, but I have slowly been learning enough about regular expressions to be able to do some useful things with FLEx.  One aspect just learned is how to use regular expressions in search and replace operations in FLEx.  (The FLEx help menus are not really very explicit on this.)

In a FLEx search and replace function — in Bulk Edit, for example — each thing that is enclosed in parentheses will return some set of results, called a capture.  You can refer to this capture with the variable $.  So the material in the first set of parentheses is $1.  The material in the second capture is $2, and so on.   Here is an example of how I used this information in the Colonial Valley Zapotec database.

Córdova normally cites a verb in the 1st person habitual.  Depending on the allomorph of the verb, the habitual of the verb might be /ti, to, te/.  The verb root will usually be four to eight letters long.  And the first person will end in /a/.

So if the form cited is tichapa, I would like it to be segmented ti+chap-a.

The "Find what" on the first line sets up a first capture group, which is the prefix, made up of t plus either e, i, or o. (Elements between square brackets are options.)  Because this whole first unit is between parentheses, it is capture group one, which I can refer to as $1 in the "Replace With" line below.

I want to replace it with the same thing, followed by a + to show the boundary.

The next capture group is a group of letters (shown by \w — meaning any wordforming character), and I have shown the number as between 4 and 8.  (On second thought, perhaps the lower number should have been three…)

Since this is the second capture group, I can refer to it by $2, in the "Replace With" line, and this time I replace it with the same thing, followed by a hyphen.

This is a first attempt at using the regular expressions with FLEx, but I think I can already see how they are going to make it possible to accomplish more sophisticated data manipulation as we try to get the Córdova diccionario into a format that we can understand.

Homograph numbers as an aid to merging entries

In a previous post I mentioned the issue of merging what are separate entries in Cordova's dictionary of Colonial Valley Zapotec.

Today I experimented with using the Homograph Numbers feature in FLEx to help me locate these entries.  In the configure columns menu, one of the options is Homograph Numbers.  I chose to restrict that entries with a homograph number greater than 0.  (Ordinary entries don't have homograph numbers; they are only assigned when two entries have identical Lexeme Forms.)

This locates about 6000 entries where there is a homograph.  In this screenshot, zèni is both 'tomar en la mano' and 'tener en la mano', so these entries should be merged.

Unfortunately, that still leaves me with lots of entries to look at -- some of them should be merged and some should not, but you need to look at the translations to decide.

I think the homograph number method would also fail to catch entries that are mostly identical, but differ by the placement of an accent, e.g. if there were an entry zéni that means the same thing as zèni.

Merging entries from Cordova's colonial Zapotec dictionary

A couple of weeks ago, I managed to import all of Cordova's 1567 dictionary of Spanish into my FLEx project.

Since that book is Spanish -- Zapotec, one thing that is revealed by created a project that can sort on the Zapotec is how many times Zapotec words are listed under multiple translations in Spanish.  This gives us a much better sense of the range of meanings of the Zapotec word.

Consider the following merged entry for aba-queta 'fall', which combines information from six different Spanish - Zapotec entries

.  

Here it would not have been too difficult to see this, since all but one are listed under caer 'fall' in Spanish.

But the value is much clearer when the Spanish glosses are more various.   Consider the following entry for aate 'harden',  where the various Spanish glosses are not alphabetically adjacent.  Nor is the idea that the same word is used for freezing and hardening obvious.

Culling blanks from the Colonial Zapotec

In the process of importing entries from Cordova dictionary of Colonial Valley Zapotec, I forgot that I needed to weed out the cases where the Zapotec Lexeme Form is blank.

This is generally because the original entry in Cordova is a cross reference.  For example:

Where we don't have a Zapotec word for the entry under Cepo de animales.  The underlying file from Thom Smith Stark's group has all of these listed as records with no Zapotec form.

That doesn't make much sense in a Zapotec- Spanish - English dictionary, so I weeded those out tonight. That results in a dictionary with 46,712 entries.  (There are still plenty of duplicates in there also...)

Here's a screenshot with the new number:

Scilicet in Cordova dictionary entries

In the Cordova dictionary of Colonial Valley Zapotec, the abbreviation scillicet means 'that is, for example'.  In the Spanish orthography of the time, this is the long s followed by a period.  In Thom Smith Stark's markup, this is {¥s[cilicet].}

Here is a screenshot of a sample Cordova entry that uses this.

Almoçadas means something like 'handfuls' in Spanish.  The FLEx entry that shows my best guess of the correct interpretation of the Cordova entry is as follows:

I think the o in front of the word xoopa is probably Spanish o 'or'.

This example shows some of the difficulties of deciphering all the information that Cordova includes in his entries!

Bulk editing Cordova Zapotec entries

Continuing on my efforts to make the enormous Cordova dictionary of colonial Zapotec.

My general goal is to have the Lexeme Form of each verb show the verb root.  Cordova's usual practice was to cite a verb in the habitual aspect for the 1st person singular.

The habitual has several allomorphs, written in the following way by Cordova (with my best guess at the intended phonemic?

• <to> /ru-/
• <ti> /ri-/
• <t> /r-/

Also sometime as <te>, though I'm not sure about /re-/ as an allomorph of the habitual in modern Valley Zapotec.

The first person is usually written as 

• <a> /=a/  after a consonant
• <ya> /=ya/ after a vowel

The version of the database that we have inherited from Thom Smith Stark often has these separated from the root as follows:

to+chìba-ya ticha-pitào,  'bendezir algo o consagrar' ('bless something or consecrate')

So the stem should be

chiba


with /to-/ and /-ya/ stripped away.  This entry also shows that the /-ya/ is not necessarily final in the entry, since Cordova often includes a typical object along with the verb.  Here the object is ticha pitào
'word of God'.

I've worked through most of the verbs in the 5000 imported Cordova entries at this point.  

My first step was to copy all the information in the original entry to the Citation Form field, so that I always have the original form available.  Then I word on the Lexeme Form field to a.) remove the habitual aspect prefixes b.) remove the 1st singular suffix, b.) put the information about completive and potential aspects into special fields.  

The procedure uses the Bulk Edit function of FLEx, generally searching for various allomorphs of the habitual and 1sg and replacing them with nothing.  This is easiest for the entries where Thom Smith Stark's analysis, where + separates the prefix , - precedes the suffix.  I can search for entries with to+, ti+, t+, -a,  and -ya pretty easily.

Here are some screen shots, first filtering to find all the examples of the pattern.  The search uses regular expressions, so ^ means at the beginning of the record and the \ makes the following + be interpreted literally as + (not some function).

Here is the bulk replace setup screen:

And here is an example of an entry after the Bulk Replace has removed the to- prefix.

(I also changed the part of speech for all items with the to+ pattern to Verb.) More difficult are the entries where Thom did not do the analysis.   It's not correct to remove every initial to sequence, since some of the resulting items are just nouns that start with to.   For example, the noun tola  'sin'  shouldn't be changed to la with a to prefix.


What I tried here was searching for the pattern to...a or to...ya, then inspecting the results to make sure that the Spanish gloss seems to be a verbal form (generally cited in either the infinitive or the past participle form).  I changed the part of speech for all of the good instances to Verb, and make a few manual changes to other parts of speech when I could figure it out. 


After the valid instances of verbal to...(y)a  were identified, I filtered the data to show only verbs and then used the same Bulk Replace method to delete the prefixes and suffixes from the Lexeme Form.

More on Importing Cordova into the FLEx database

For many years, Thom Smith Stark and his students worked on an electronic version of the giant Cordova dictionary of Colonial Valley Zapotec.  I've been working for the last few days on the process for importing this into FLEx.

I have a few different versions of the electronic Cordova.  One is a MS-Access database, and the advantage of this version is that each of the multiple Zapotec words listed as the translation of the Spanish gets its own record.  In the design of the MS-Access, the ID number is keyed to the entry in the Cordova dictionary and there are two linked tables. (One with entry, folio number and Spanish, the other with the Zapotec, notes, etc.) To see both you construct a MS-Access query, which looks like this:

This can be output as an Excel file.  (Then from Excel =--> Sheet swiper --> FLEx.)

After the export to Excel, you need to add a row to the top with \lx over the column that will be the Lexical Form and \gn (=Gloss National) for the Spanish.  The original database has a Folio field that tells you what page of the book the word is located on.  I called this field \cordova page number. And for all the rows, I added a field \source with the value [Cordova import].

In order to get Sheet swiper to work properly, the \lx column needs to be first.  So I reordered the Excel columns.  I also sorted the Zapotec column to remove blanks.  (In the original book, these are cross-references from one Spanish entry to another.)

The original database also has two Zapotec fields, one with diacritics (ZAP_COMP) and one without (ZAP).  I decided not to import the version without diacritics, so I didn't put a backslash code over that column in the Excel.  The result looks something like this

Then I ran Sheet Swiper, which converts the Excel file into a standard format dictionary file.  That is fairly easy.

Within FLEx, you use the Lexicon | Import | Standard Format lexical data dialogue to pull this into FLEx.

You have to go through a few steps here.  Most are pretty easy to understand, but a few are not completely intuitive.  In the old Shoebox/Toolbox format the languages were divided into Vernacular, National, Regional, English.  I had used Spanish as the equivalent of National in earlier versions, and that is why I put the \gn tag over that column.  So in the dialogues, you need to tell FLEx that for this project National = Spanish.

For the various fields, you also need to tell FLEx where they will go in the entry.  If it doesn't know where they go, it will put the information in a field called "Import Residue".  That's okay, since it doesn't lose the information, but it is better to specify where it will go, if you know.  I wanted the Folio number to go in the Source field in each entry so I specified it in that way.

Running through this all let me import the first 483 test entries from Cordova to FLEx.

Within FLEx, I also made a few global changes.  Since the Zapotec form listed has all kinds of information in it, I wanted to segregate this out into other fields and work towards having just the root of the word as the Lexeme Form.  But because I didn't want to lose any information, I copied all of the information from the original entry into the Citation Form field via the Bulk Edit Entries dialogues.  A few examples:

The Zapotec field has information about things that are corrected somewhere (presumably in the pages of errata at the beginning.)  This entry has a correction in the original.  I changed the Lexical Form to reflect the correction and show the original form + correction in the Citation Form field.

The original also contains information about the prefix that a verb takes in the completive and potential aspects.  (The form is cited in the habitual.)  In the TSS database, this follows the field marker /cv/

This entry shows how the information got processed.  I created custom fields for the completive, potential, and habitual.  Then I filtered the imported entries to find /cv/ and used the "Click copy" feature in Bulk Edit to populate these fields.  After the fields had been populated, I used the "Delete" feature in Bulk Edit to remove this from the Lexeme Form:

Searching Whitecotton

One of the challenges of working with Colonial Zapotec is that although there is an enormous dictionary, it is a Spanish-to-Zapotec dictionary, in only a vague approximation of alphabetical order.  Joe and Ruth Bradley Whitecotton produced a very valuable resource in 1993 when they produced a Zapotec-to-Spanish reverse version.  This is much better for trying to go from Zapotec to Spanish, but still there are problems --

Zapotec verbs normally have a prefix, so verb stems might be distributed among different aspects, relativized forms, nominalized forms, etc.  Nouns might appear alone, but also as parts of compounds or in variant spellings. It would be nice to be able to find all of the relevant forms.   People at UCLA produced a PDF version of Whitecotton (which is otherwise hard to obtain), but this is an image file and not searchable.

After a little fiddling, I was able to apply optical character recognition to this PDF, resulting for the first time in a searchable version of the Whitecotton and Whitecotton Colonial Zapotec -- Spanish dictionary.  In the first afternoon of having it available, I'm already finding words with more ease.

Title page for the Whitecotton and Whitecotton dictionary

PDF version, now processed via OCR and searchable