Learning from Sparse Data

Size: px

Start display at page:

Download "Learning from Sparse Data"

Amberlynn Morris
5 years ago
Views:

1 Learning from Sparse Data MT & Bible Translation Jon Riding United Bible Societies

The Problem Space 7000+ natural languages Language

2 The Problem Space natural languages Language Independent Systems Diverse genres, poetry, rhetoric Lay translators 2

3 The Translation Task The narrative of the text The narrative of receptor language & culture Meaning emerges from this encounter 3

4 Focus on Analysis Key term glossing Interlinear back-translation Morphology analysis Spelling checks etc 4

5 Mainstream MT Focus on synthesis Genre specifc Needs big data to train from Language pair specifc 5

6 Bible Translation Realities Little or no training data Existing systems cannot contribute until later in a project Non-technical translation staff 6

7 Introducing parsebot: Discrete processes Supporting one another Aggregating results 7

8 A Learning Framework Playing to your strengths Doing more with less Improving current systems Facilitating new systems Supporting translators better 8

9 Playing to strengths The Translation Team Mother tongue speakers Deep but possibly unstructured knowledge of target language Base/Model Texts LWC/Gateway Languages Lexica, Encyclopedia, etc 9

10 Limitations of Learning Machines Humans learn diachronically Machines need data to learn from Machines trade learning over time for learning from breadth No breadth, no learning... 10

11 How to help the machine Subset the task: Learn glosses or Morphology or Names etc Not so good at putting the pieces together 11

12 Signal and Noise Machine Learning Curve Single context learning: Glossing, morphology etc... More data, good learning Little data, poor learning 12

13 Language as a whole Human beings don t learn glosses, or morphology or syntax We just learn language, one bite at a time Over time, bite-sized learning becomes a coherent whole 13

14 Learning the patterns Human Learning: Hugely parallel Many contexts Reinforcement across different contexts Can learn much from little 14

15 Signal and Noise Learning Much from Little Signal/Noise Ratio is not proportional to amount of data Some signals may be strong in a smaller context Early learning improves the SNR as the amount of data grows 15

16 Modelling sparse data learning Finding patterns The parsebot Cluster: Discreet processes each exploring a particular context: Glossing, morphology, close cognates, names, syntax etc Reinforcement Learning 16

17 parsewindow Sets the learning context Dynamic sizing Local learning 17

18 Language Model Records all parses Aggregates local parses into global tables Exploits local parses across the wider data set 18

19 Supervision Dialogue Maintains a list of current interaction requests Invites user confrmation of parses Limited to binary questions 19

20 Benefts for Translators Contributes from the start of a project Brings current systems on-line much earlier in a project Encourages the user to regularly review work in hand Limits time needed to work through long lists of items at checking sessions Lets the machine do the tedious stuff 20

21 Example data: Target Text: Latin Vulgate Jn Base Text: UBS Greek NT Lemmatised 1 In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum. 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita erat lux hominum 5 et lux in tenebris lucet et tenebrae eam non conprehenderunt 6 fuit homo missus a Deo cui nomen erat Iohannes 7 hic venit in testimonium ut testimonium perhiberet de lumine ut omnes crederent per illum 8 non erat ille lux sed ut testimonium perhiberet de lumine 9 erat lux vera quae inluminat omnem hominem venientem in mundum 10 in mundo erat et mundus per ipsum factus est et mundus eum non cognovit. 21

22 Example botset: MorphBot non-concat morph analyser NameBot CogNomen name fnder CogBot Close cognate form fnder StemBot Radix fnder GlossBot Parallel gloss analyser?bot Set is extensible... 22

23 Bot Reports: vv 1-3 CogBot ipso ipsum? facta factum? In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 1 MorphBot _um? Note: Each analysis reinforces the other 23

24 Bot Reports: vv 1-4 stembot de_? ips_? fact_? In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita erat lux hominum 1 24

25 Bot Reports: vv 1-5 In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita erat lux hominum 5 et lux in tenebris lucet et tenebrae eam non conprehenderunt 1 CogBot tenebrae tenebris? Note: awaiting confrmation... 25

26 Bot Reports: vv NameBot: Iohannes Ἰωάννης? CogBot: non nomen? glossbot: de* θε*? In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita erat lux hominum 5 et lux in tenebris lucet et tenebrae eam non conprehenderunt 6 fuit homo missus a Deo cui nomen erat Iohannes 26

27 Bot Reports: vv omnia per ipsum facta sunt et CogBot: sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita omnes omnia? erat lux hominum 5 et lux in StemBot tenebris lucet et tenebrae eam non e_t? conprehenderunt 6 fuit homo Note: stembot is right to missus a Deo cui nomen erat Iohannes 7 hic venit in testimonium observe the relationship ut testimonium perhiberet de between est and erat even if lumine ut omnes crederent per we might not immediately think illum of it like that. 27

28 Bot Reports: vv 1-8 CogBot: hic hoc? ille illum? 1 In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita erat lux hominum 5 et lux in tenebris lucet et tenebrae eam non conprehenderunt 6 fuit homo missus a Deo cui nomen erat Iohannes 7 hic venit in testimonium ut testimonium perhiberet de lumine ut omnes crederent per illum 8 non erat ille lux sed ut testimonium perhiberet de lumine 28

29 Bot Reports: vv 1-9 CogBot: hominem hominum? StemBot: homin_ 4 in ipso vita erat et vita erat lux hominum 5 et lux in tenebris lucet et tenebrae eam non conprehenderunt 6 fuit homo missus a Deo cui nomen erat Iohannes 7 hic venit in testimonium ut testimonium perhiberet de lumine ut omnes crederent per illum 8 non erat ille lux sed ut testimonium perhiberet de lumine 9 erat lux vera quae in luminat omnem hominem venientem in mundum 29

30 Bot Reports: vv 1-10 CogBot: factus (facta, factum)? mundo mundum mundus? StemBot: mund_? GlossBot: fact* ποι* mund* κοσμ* 1 In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum 2 hoc erat in principio apud Deum 3 omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est 4 in ipso vita erat et vita erat lux hominum 5 et lux in tenebris lucet et tenebrae eam non conprehenderunt 6 fuit homo missus a Deo cui nomen erat Iohannes 7 hic venit in testimonium ut testimonium perhiberet de lumine ut omnes crederent per illum 8 non erat ille lux sed ut testimonium perhiberet de lumine 9 erat lux vera quae inluminat omnem hominem venientem in mundum 10 in mundo erat et mundus per ipsum factus est et mundus eum non cognovit. 30

31 Synopsis: 20 Close Cognates: 7 Morphology templates 9 Stem radices 3 Glosses 1 Proper name 31

32 Conclusion This is a strong model for learning about a developing text You can learn a lot from not much data The data learnt are key to bringing other systems on line early in a project such as: Glosser, Spelling checker Interlinear 32

33 33

Common Core Correlations Grade 11

Common Core Correlations Grade 11 Number ELACC11-12RL1 ELACC11-12RL2 ELACC11-12RL3 ELACC11-12RL4 Reading Literary (RL) Grades Eleven/Twelve Key Ideas and Details Cite strong and thorough textual evidence