Rasa NLU:Entity Synonyms detection inconsistency - rasa-nlu

Question

Me and my team have been using Rasa NLU as a replacement for MS LUIS for over 2 months now, and it has worked out pretty well for us so far. Now we have around 900 entries as Entity Synonyms(as we were using List entity in LUIS).
And only for some utterances, the entity is detected as synonyms and for the majority of utterances, it is unable to detect Entity Synonyms. In order to detect synonyms, I have to create another simple entity which again we are manually training with all the synonym values, once the intents are trained with this simple Entity Rasa seems to detect entity for this intent as both simple and synonyms.
And another quick question, Is the Entity Synonyms in Rasa designed to return only one matched entity(unlike LUIS which used to return all the matched entities values)?
Is there any alternative to list entity from LUIS here in Rasa?

Answer 1

Entity Synonyms in Rasa can lead to some confusion. The actual functionality that they provide is very simple. For each entity that is parsed by the model the value of that entity is checked against the list of entity synonyms. If the value matches an entity synonym then it is replaced with the synonym value.
The big catch in the above statement is that the the entity has to be identified by the model before it can be replaced with a synonym.
So take this as a simplified example. Here is my entity synonym definition:
{
"value": "New York City",
"synonyms": ["NYC", "nyc", "the big apple"]
}
If my training data only provides this example:
{
"text": "in the center of NYC",
"intent": "search",
"entities": [
{
"start": 17,
"end": 20,
"value": "New York City",
"entity": "city"
}
]
}
It is very unlikely that my model will be able to detect an entity in a sentence like In the center of the big apple. As I said above if the big apple isn't parsed as an entity by the model it cannot be replaced by the entity synonyms to read New York City.
For this reason you should include more examples in the actual common_examples of the training data with the entities labeled. Once all of the variations of the entity are being classified correctly then add those values to the entity synonym and they will be replaced.
[
{
"text": "in the center of NYC",
"intent": "search",
"entities": [
{
"start": 17,
"end": 20,
"value": "New York City",
"entity": "city"
}
]
},
{
"text": "in the centre of New York City",
"intent": "search",
"entities": [
{
"start": 17,
"end": 30,
"value": "New York City",
"entity": "city"
}
]
}
]
I've opened a pull request into the Rasa docs page to add a note to this effect.

Answer 2

Firstly, I have downloaded some LUIS model JSON for doing this, as shown in the following screenshot:
Next, I have written a sample C# console app for converting LUIS Model Schema into RASA.
Here is the LUISModel model class.
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
namespace JSONConversion.Models
{
public class LuisSchema
{
public string luis_schema_version { get; set; }
public string versionId { get; set; }
public string name { get; set; }
public string desc { get; set; }
public string culture { get; set; }
public List<Intent> intents { get; set; }
public List<entity> entities { get; set; }
public object[] composites { get; set; }
public List<Closedlist> closedLists { get; set; }
public List<string> bing_entities { get; set; }
public object[] actions { get; set; }
public List<Model_Features> model_features { get; set; }
public List<regex_Features> regex_features { get; set; }
public List<Utterance> utterances { get; set; }
}
public class regex_Features
{
public string name { get; set; }
public string pattern { get; set; }
public bool activated { get; set; }
}
public class Intent
{
public string name { get; set; }
}
public class entity
{
public string name { get; set; }
}
public class Closedlist
{
public string name { get; set; }
public List<Sublist> subLists { get; set; }
}
public class Sublist
{
public string canonicalForm { get; set; }
public List<string> list { get; set; }
}
public class Model_Features
{
public string name { get; set; }
public bool mode { get; set; }
public string words { get; set; }
public bool activated { get; set; }
}
public class Utterance
{
public string text { get; set; }
public string intent { get; set; }
[JsonProperty("entities")]
public List<Entities> Entities { get; set; }
}
public class Entities
{
[JsonProperty("entity")]
public string Entity { get; set; }
public int startPos { get; set; }
public int endPos { get; set; }
}
}
Here is the RASAModel model class:
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
namespace JSONConversion.Models
{
public class RASASchema
{
public Rasa_Nlu_Data rasa_nlu_data { get; set; }
}
public class Rasa_Nlu_Data
{
public List<Entity_Synonyms> entity_synonyms { get; set; }
public List<Regex_Features> regex_features { get; set; }
public List<Common_Examples> common_examples { get; set; }
}
public class Entity_Synonyms
{
public string value { get; set; }
public List<string> synonyms { get; set; }
}
public class Common_Examples
{
public string text { get; set; }
public string intent { get; set; }
public List<Entity> entities { get; set; }
}
public class Entity
{
public string entity { get; set; }
public string value { get; set; }
public int start { get; set; }
public int end { get; set; }
}
public class Regex_Features
{
public string name { get; set; }
public string pattern { get; set; }
}
}
And I have written 2 methods which parse the LUISModel model class for synonyms from the phraselist section and adds them in the common_examples object in RASA_NLU training object.
using JSONConversion.Models;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
namespace JSONConversion.Services
{
public static class JSONHelper
{
public static Task<string> ReadFromFile(string FilePath)
{
try
{
Task<string> readFromFileTask = Task.Run<string>(() =>
{
return File.ReadAllText(FilePath);
});
return readFromFileTask;
}
catch(Exception ex)
{
throw;
}
}
public static RASASchema ConvertLUISJSON(string StringifiedLUISJson)
{
try
{
LuisSchema luisSchema = JsonConvert.DeserializeObject<LuisSchema>(StringifiedLUISJson);
RASASchema rasaSchema = new RASASchema();
rasaSchema.rasa_nlu_data = new Rasa_Nlu_Data();
rasaSchema.rasa_nlu_data.common_examples = new List<Common_Examples>();
rasaSchema.rasa_nlu_data.entity_synonyms = new List<Entity_Synonyms>();
rasaSchema.rasa_nlu_data.regex_features = new List<Regex_Features>();
luisSchema.closedLists.ForEach(x =>
{
x.subLists.ForEach(y =>
{
rasaSchema.rasa_nlu_data.entity_synonyms.Add(new Entity_Synonyms()
{
value = y.canonicalForm,
synonyms = y.list
});
});
});
luisSchema.model_features.ForEach(x =>
{
rasaSchema.rasa_nlu_data.entity_synonyms.Add(new Entity_Synonyms()
{
value = x.name,
synonyms = x.words.Split(',').ToList()
});
});
luisSchema.regex_features.ForEach(x =>
{
rasaSchema.rasa_nlu_data.regex_features.Add(new Regex_Features()
{
name = x.name,
pattern = x.pattern
});
});
luisSchema.utterances.ForEach(x =>
{
Common_Examples rasaUtterances = new Common_Examples();
rasaUtterances.text = x.text;
rasaUtterances.intent = x.intent;
List<Entity> listOfRASAEntity = new List<Entity>();
x.Entities.ForEach(y =>
{
listOfRASAEntity.Add(new Entity()
{
start = y.startPos,
end = y.endPos,
entity = y.Entity,
value = x.text.Substring(y.startPos, (y.endPos - y.startPos) + 1)
});
});
rasaUtterances.entities = listOfRASAEntity;
rasaSchema.rasa_nlu_data.common_examples.Add(rasaUtterances);
});
return rasaSchema;
}
catch (Exception ex)
{
throw;
}
}
}
}
And just called those JSON conversion methods to convert LUIS Models into RASA models.
using System.Text;
using JSONConversion.Services;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
namespace JSONConversion
{
class Program
{
static void Main(string[] args)
{
string json = JsonConvert.SerializeObject(JSONConversion.Services.JSONHelper.ConvertLUISJSON(JSONHelper.ReadFromFile(#"C:\Users\xyz\Documents\luis.json").Result), new JsonSerializerSettings()
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
Formatting = Formatting.Indented
});
File.WriteAllText(#"C:\Users\xyz\Desktop\RASA\data\examples\RasaFormat.json", json, Encoding.UTF8);
}
}
}
After getting the RASA model, you can just simply train RASA for synonyms.