I have massive amounts of natural language questions in the format of "Subject-entity [tab] relationship [tab] Object-entity [tab] question" as follows:
www.freebase.com/m/01jp8ww www.freebase.com/music/album/genre www.freebase.com/m/01qzt1 Which genre of album is harder.....faster?
www.freebase.com/m/0np6z99 www.freebase.com/music/album/release_type www.freebase.com/m/02lx2r what format is fearless
www.freebase.com/m/0wzc58l www.freebase.com/people/person/place_of_birth www.freebase.com/m/0n2z what city was alex golfis born in
www.freebase.com/m/0jtw9c www.freebase.com/film/writer/film www.freebase.com/m/05szq8z what film is by the writer phil hay?
www.freebase.com/m/0gys2sn www.freebase.com/people/deceased_person/place_of_death www.freebase.com/m/0tzls Where did roger marquis die
www.freebase.com/m/01fwty www.freebase.com/people/deceased_person/cause_of_death www.freebase.com/m/0gk4g what was the cause of death of yves klein
www.freebase.com/m/02cft www.freebase.com/location/location/people_born_here www.freebase.com/m/0k8n4c1 Which equestrian was born in dublin?
www.freebase.com/m/01htzx www.freebase.com/tv/tv_genre/programs www.freebase.com/m/04g1jv6 What is a tv action show?
How does one parse these natural language question files to obtain the answer to the question? I am planning on using ruby to parse these millions upon millions of questions, although I am a bit stuck on how to actually obtain the answers to the questions in the files from freebase.
From what I have viewed from the direct links, there doesn't to seem to actually be an answer to the question located in them, so is there some other method to obtaining the answers?
Related
I would like you to leave your comments on: If it is right to say that any question having the term ‘softmax function’ is a duplicate copy to the other questions having the term ‘softmax function’ on it?
I think that it is wrong to say that any question with the term “Softmax function” is a duplicate copy question. Why? Because from the Cambridge Dictionary, the meaning of the term ”duplicate copy” is define as “been an exact copy of something or something that is an exact copy of something else”.
Diversity of opinions, questions and answers on the term ”softmax function” should be well come and should not be called duplicate copy. But, instead should be called related, and grouped under related questions.
For more information on how to apply softmax in machine learning, please click the link below:
How to use softmax output in python for neural-network and machine-learning to interpret Multinomial Logit Model?
We are uploading a PDF with semi structured Question & Answers. QnA maker is merging same question if they are successive. If there is some other question exist between them, then QnA maker is not merging same questions. For example.
Q Machine was not able to be started
Answer 1
Q Machine was not able to be started
Answer 2
Q Burning plastic smell on machine
Answer 3
Now the QnA Maker will train it like this
Q Machine was not able to be started
Answer 1
Answer 2
Q Burning plastic smell on machine
Answer 3
Why is QnA is behaving like this and how to separate same questions. Help is required.
This is expected behavior. QnA works on the 'one question (and related, similar questions)' to one answer idea, and expects unique questions for the queries. The QnA documentation states:
A knowledge base consists of question and answer (QnA) sets. Each set has one answer and a set contains all the information associated with that answer. An answer can loosely resemble a database row or a data structure instance.
The required settings in a question-and-answer (QnA) set are:
a question - text of user query, used to QnA Maker's machine-learning, to align with text of user's question with different wording but the same answer
the answer - the set's answer is the response that's returned when a user query is matched with the associated question
Each set is represented by an ID.
The optional settings for a set include:
Alternate forms of the question - this helps QnA Maker return the correct answer for a wider variety of question phrasings
Metadata: Metadata are tags associated with a QnA pair and are represented as key-value pairs. Metadata tags are used to filter QnA pairs and limit the set over which query matching is performed.
Multi-turn prompts, used to continue a multi-turn conversation
QnA Maker doesn't differentiate between the two questions because it isn't two questions. It's literally the same question with two different answers.
This particular case would be a good use of QnAMaker's multi-turn prompt feature, where, after the customer has put in the query 'The machine won't start', QnA can follow up with a prompt that says "Which machine did you mean? Machine A or Machine B", and whichever they choose leads to the correct answer. I would look into Multi-Turn Conversations for your knowledgebase.
Here is an example of the data I am using:
"Q7: How does gender income inequality manifest in the communities in which you live and/or work? What do you believe is needed to help close the wealth gap between men and women as well as among women of different races in the county?
Wouldn’t go into pay equity because that would get dismissed. The wealth gap is a more compelling argument.
Equity is more emotional. And wealth gap is more numbers. It goes toward the same thing though."
I am running Rstudio and using library(RQDA) and library(tidyverse).
I am trying to analyze several qualitative interviews formatted in question/answer form as in the provided example. I finished the coding process and now I'm trying to find themes. While coding, I created code categories that correspond with each interview question with the hopes that I would be able to pull out all the codings per code category now. Unfortunately, I cannot figure out how to do it and would appreciate some assistance!
thanks
I am not sure about this but I understood code categories to be helpful for structuring your work by your theoretical perspective. If you create a code category for each interview question (i.e., the topic of the question is your theme/code category), you may have various codes belonging to one "code category", which might not have that much in common. Alternatively, you could create cases (case 1 might be the answers to the first interview question, etc.): "b) Open a file, select part of the file, select a case name, then click button "Link" in "Cases" tab, you can thus link the selected part of file to the selected case" (http://rqda.r-forge.r-project.org/documentation_2.html).
I have tags on my website, and I input them one by one when I create a blog post. I love gmail's new feature, that ask you if you want to include X in a mail, if you type Y's name and that you often include both of them in the same messages.
I'd like to do something similar on my website, but I don't know how to represent the tags "related-ness" in an object or database ... thoughts ?
It all boils down to create associations between certain characteristics of your posts and certain tags, and then - when you press the "publish" button - to analyse the new post and propose all tags matched with your post characteristics.
This can be done in several ways from a "totally hard-coded" association to some sort of "learning AI"... and everything in-between.
Hard-coded solutions
This are the simplest algorithms to implement. You should first decide what characteristics of your post are relevant for tagging (e.g.: it's length if you tag them "short" or "long", the presence of photos or videos if you tag them "multimedia-content", etc...). The most obvious is however to focus on which words are used in posts. For example you could build a mapping like this:
tag_hint_words = {'code-development' : ['programming',
'language', 'python', 'function',
'object', 'method'],
'family' : ['Theresa', 'kids',
'uncle Ben', 'holidays']}
Then you would check your post for the presence of the words in the list (the code between [ and ] ) and propose the tag (the word before :) as a possible candidate.
A common approach is to give "scores", or in other word to put a number that indicates the probability a given tag is the right one. For example: if your post would contain the sentence...
After months of programming, we finally left for the summer holidays at uncle Ben's cottage. Theresa and the kids were ecstatic!
...despite the presence of the word "programming" the program should indicate family as the most likely tag to use, as there are many more words hinting.
Learning AI's
One of the obvious limitations of the above method is that - say one day you pick up java beside python - you would probably need to change your code and include words like "java" or "oracle" too. The same applies if you create new tags.
To circumvent this limitation (and have some fun!!) you could try to implement a learning algorithm. Learning algorithms are those who refine their outcome the more you use them (so they indeed... learn!). Some algorithm requires initial training (many spam filters and voice recognition programs need this initial "primer"). Some don't.
I am absolutely no expert on the subject, but two common AI's are: the Naive Bayes Classifier and some flavour of Neural network.
Although the WP pages might look scary, they are surprisingly easy to implement (at least in Python). Here's the recording of a lecture at PyCon 2009 on the subject "Easy AI with Python". I found it very informative and even somehow inspiring! :)
HTH!
You should have a look at this post :
Any suggestions for a db schema for storing related keywords?
If you're looking for a schema for storing related tags it will help.
Relevancy searches where multiple agents play a part are usually done using Collaborative filtering. You might want to give that a look see.
Look up Clustering (Machine Learning algorithm). Don't be intimidated by math, it's a pretty straightforward algorithm. Check out Machine Learning for Hackers for simpler explanations of many Machine Learning algorithms and methods.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Every once in a while I'm confronted with displaying a list of available languages, and each and every time I ask my self:
Is it better to display the language in:
the currently selected language
English
in the language according to the button/list item
Examples:
English
German
French
or
English
Deutsch
Français
Is there any convention on which one should be used, is more polite or better in any other way? Are there other options?
I would say it's best to display the language in "its own language" (option #3). You can not necessarily expect the user to know the currently selected language, nor expect him to know English.
What's tricky is how to display the "Select your language" button in a language neutral way. I usually go for a flag indicating the current language since that tends to get the message across eventhough there's not always a 1:1 mapping between country and languages.
I definitely think you should display in the language that matches the item in the button list.
Reasons:
If it's not the language you're interested in, you won't mind if you don't understand it, as long as you can find your own language.
Think about the last time you called customer service. How many times have you heard something like, "Para Espanol, marque dos"? It's very common, accepted practice to mix different languages in one UI (whether visual or audible).
Think about how you'd feel if you went to a Spanish site, and you couldn't find your language under "E". Maybe, eventually, you'd notice "Ingles", and think it probably translated to "English", but it's definitely better to save the user the trouble of translating and mentally alphabetizing.
The standard (in both senses of the word, i.e. what is actually used in the real world, and what the IETF/W3C/ISO says) is to use ISO 639-1 Alpha-2 language codes. Maybe augmented with either the full name of the language in English, the language itself, a romanic transliteration of the name in the language itself or any combination thereof.
So, to keep with your example:
[de] German - Deutsch
[en] English
[fr] French - Français
[ja] Japanese - 日本語 (Nihongo)
Two options, first the name of the language in the selected locale or English, then the name of the language in itself between parens, or the other way around, e.g.:
English
French (Français)
German (Deutsch)
Spanish (Español)
or
English
Français (French)
Deutsch (German)
Español (Spanish)
English language name:
Pros:
Predictable sorting.
No need to think about different text flows.
Cons:
Users who doesn't speak English might have ha harder time finding their language.
If the rest of the application is translated, it might look sloppy or grammatically wrong: Ditt språk är English/votre langue est English.
Language in its own name:
Pros:
Easier for the non-English speaker.
You have to think about encoding and text flow; A useful exercise. :-)
Cons:
Harder to navigate if the user is used to English or has her mind set on finding an English name.
You have to consider all language variants.
What is right really depends on the rest of your application. You might want to consider having all language names translated to all languages. If english is choosen, then you get to pick from:
English
Swedish
French
If Swedish:
Engelska
Svenska
Franska
...and French:
Anglais
Suédois
Français
But then the translantion problem has turned from O(n) to O(n^2), which might be acceptable depending on what your current value of n is.
EDIT
As deceze points out. you will also have to handle the case when a user accidentally switches to a language she doesn't understand, and provide a way back - for example by always including a few major languages.
I find it harder to find "Magyar" in a list of languages.
Because there are languages with non-latin character set, this is not a simple first-letter-lookup, as I lose focus when I first meet one of these.
Where should I look? At 'M' - Magyar? But where is M? EDIT: M in the (current language's) alphabet, not on the keyboard.
Have a look at this (from Wikipedia):
Български - I know, this is Bulgarian, but
བོད་ཡིག - what is this?
Bosanski
Català
Česky
Dansk
Deutsch
Ελληνικά
I would prefer something like this:
A...
B...
C...
.
.
Hungarian (Magyar)
If the UI was Japanese, I would ctrl+f-ing "Magyar", though.
Whatever you do don't use the IP location to set the language.
Google is very annoying about this -- when logging on from a new location I get google in the local language and script. This is really annoying particularly, anywhere southeast of Croatia.
The worse offender though is Microsoft. When trying to purchase software thier servers keep switching languages depending on your location and in many cases makes it impossible pay for anything by Credit Card as the addresses and zip codes etc. are validated in the local format and not where your credit card was actually issued. ( By the way MS the first four digits of a credit card number indicate the issuing institution which is tied to a particular country so its not rocket science to work out a UK postcode format is required rather than say a six digit german ZIP code.
Use country-flags in combination with the language name in that language (Deutsch, Francais, Nederlands, ...).
I don't know about any programming related conventions about this but i would prefer to see the name of a language in its own language.
For example:
English
Türkçe
Deutsch
Have a look at your Regional Settings.
This is how Microsoft implemented it. Seems like your version 1.
alt text http://www.freeimagehosting.net/uploads/1c14f9f60d.jpg