Semantic Space

One of the projects I'm working on is quantifying how abstract concepts like "freedom" or "business" differ between cultures using the property listing task (PLT) of cognitive psychology. An important part of that is being able to compute the similarity between two words. For example, we would expect the similarity between "dog" and "cat" to be higher than the similarity between "dog" and "atom". People in the field of natural language processing (NLP) have been working on this problem for years. Recently, with the rise of AI, some very good tools for this have emerged. In general, these tools are called "transformers". They turn words and phrases into points in space. Once they're in space, you can see how close together or far apart they are. Today I spent the day reading about these transformers and playing around with them.

The transformer I used today is only about 1.5 years old--so pretty new. What's interesting about this transformer is that it has been trained on over 100 currently living languages. So, no only will it say how similar "dog" and "cat" are, but how similar "perro" and "gato" are. Even better, it will say how similar "dog" and "gato" are, or "perro" and "cat".

Here's a multilingual set of words and the similarity scores. In the (dog, dog) slot we have a 1.0, which is perfectly similar. In the (dog, perro) slot, we see 0.822, not perfect.

	dog	perro	cat	gato	cow	vaca
dog	1.000000	0.822781	0.648068	0.653384	0.643983	0.492068
perro	0.822781	1.000000	0.621134	0.639385	0.632134	0.499414
cat	0.648068	0.621134	1.000000	0.790680	0.589765	0.444248
gato	0.653384	0.639385	0.790680	1.000000	0.582464	0.510511
cow	0.643983	0.632134	0.589765	0.582464	1.000000	0.504098
vaca	0.492068	0.499414	0.444248	0.510511	0.504098	1.000000

Why would (dog, perro) not be 1.0 also? Well, while both refer to dogs, in english "dog" has different sense clusters, e.g. to nag, to pursue relentlessly in English, whereas in Spanish "perro" is never used as a verb at all. These different contexts pull the words apart in semantic space and reduce their similarity.

 

This demonstrates the importance of context. If we change the words to simple phrases:

 

	animal: dog	animal: perro	animal: cat	animal: gato	animal: cow	animal: vaca
animal: dog	1.000000	0.917744	0.869536	0.842130	0.839884	0.816267
animal: perro	0.917744	1.000000	0.825918	0.811056	0.810678	0.785195
animal: cat	0.869536	0.825918	1.000000	0.903485	0.815107	0.757159
animal: gato	0.842130	0.811056	0.903485	1.000000	0.790497	0.778824
animal: cow	0.839884	0.810678	0.815107	0.790497	1.000000	0.817964
animal: vaca	0.816267	0.785195	0.757159	0.778824	0.817964	1.000000

 

We see that (animal:dog, animal:perro) is now 0.91. Adding "animal" improves the behavior of the entire set of similarities.

 

We hope to see similar performance applying this to the PLT data for abstract concepts. The idea is to ask people to describe an abstract concept like "freedom", to build sense clusters, and quantify shifts in the sense clusters across languages and cultures. More to come.

☙❧

  

I spent most of today working on this stuff, but we did take a walk along the beach path today.

Near the navy college there is a row of retired heavy guns pointing into the Pacific. Ironically, up on the hill you can see the modern navy hospital. It turns out that two of my collaborator's daughters were born there, though not the daughter we met at lunch on Saturday.

 

Also, here's the obligatory sunset photo: