Full Text Search in Linq - linq

Question

There's no full text search built into Linq and there don't seem to be many posts on the subject so I had a play around and came up with this method for my utlity class:
public static IEnumerable<TSource> GenericFullTextSearch<TSource>(string text, MyDataContext context)
{
//Find LINQ Table attribute
object[] info = typeof(TSource).GetCustomAttributes(typeof(System.Data.Linq.Mapping.TableAttribute), true);
//Get table name
String table = (info[0] as System.Data.Linq.Mapping.TableAttribute).Name;
//Full text search on that table
return context.ExecuteQuery<TSource>(String.Concat("SELECT * FROM ", table, " WHERE CONTAINS(*, {0})"), text);
}
And added this wrapper to each partial Linq class where there is a full text index
public static IEnumerable<Pet> FullTextSearch(string text, MyDataContext context)
{
return (LinqUtilities.GenericFullTextSearch<Pet>(text, context) as IEnumerable<Pet>);
}
So now I can do full text searches with neat stuff like
var Pets = Pet.FullTextSearch(helloimatextbox.Text, MyDataContext).Skip(10).Take(10);
I'm assuming only a very basic search is necessary at present. Can anyone improve on this? Is it possible to implement as an extension method and avoid the wrapper?

Answer 1

The neatest solution is to use an inline table valued function in sql and add it to your model
http://sqlblogcasts.com/blogs/simons/archive/2008/12/18/LINQ-to-SQL---Enabling-Fulltext-searching.aspx
To get it working you need to create a table valued function that does
nothing more than a CONTAINSTABLE query based on the keywords you
pass in,
create function udf_sessionSearch
(#keywords nvarchar(4000)) returns table as return (select [SessionId],[rank]
from containstable(Session,(description,title),#keywords))
You then add this function to your LINQ 2 SQL model and he presto you
can now write queries like.
var sessList = from s in DB.Sessions
join fts in DB.udf_sessionSearch(SearchText) on s.sessionId equals fts.SessionId
select s;

Answer 2

I was pretty frustrated with the lack of clear examples... especially when there are potentially large data sets and paging is needed. So, here's an example that hopefully encompasses everything you might need :-)
create function TS_projectResourceSearch
( #KeyStr nvarchar(4000),
#OwnId int,
#SkipN int,
#TakeN int )
returns #srch_rslt table (ProjectResourceId bigint not null, Ranking int not null )
as
begin
declare #TakeLast int
set #TakeLast = #SkipN + #TakeN
set #SkipN = #SkipN + 1
insert into #srch_rslt
select pr.ProjectResourceId, Ranking
from
(
select t.[KEY] as ProjectResourceId, t.[RANK] as Ranking, ROW_NUMBER() over (order by t.[Rank] desc) row_num
from containstable( ProjectResource,(ResourceInfo, ResourceName), #KeyStr )
as t
) as r
join ProjectResource pr on r.ProjectResourceId = pr.ProjectResourceId
where (pr.CreatorPersonId = #OwnId
or pr.ResourceAvailType < 40)
and r.row_num between #SkipN and #TakeLast
order by r.Ranking desc
return
end
go
select * from ts_projectResourceSearch(' "test*" ',1002, 0,1)
Enjoy,
Patrick

Answer 3

I use a little hack using Provider Wrapper techniques. I have a c# code that rewrite magic word in SQL with FTS search for MS SQL (you can adjust for any server you like).
if you have context class MyEntities, create subclass like
public class MyEntitiesWithWrappers : MyEntities
{
private IEFTraceListener listener;
public string FullTextPrefix = "-FTSPREFIX-";
public MyEntitiesWithWrappers(): this("name=MyEntities")
{
}
public MyEntitiesWithWrappers(string connectionString)
: base(EntityConnectionWrapperUtils.CreateEntityConnectionWithWrappers(connectionString,"EFTracingProvider"))
{
TracingConnection.CommandExecuting += RewriteFullTextQuery;
}
/// <summary>
/// Rewrites query that contains predefined prefix like: where n.NOTETEXT.Contains(Db.FullTextPrefix + text) with SQL server FTS
/// To be removed when EF will support FTS
/// </summary>
/// <param name="o"></param>
/// <param name="args"></param>
public void RewriteFullTextQuery(object o, CommandExecutionEventArgs args)
{
var text = args.Command.CommandText;
for (int i = 0; i < args.Command.Parameters.Count; i++)
{
DbParameter parameter = args.Command.Parameters[i];
if (parameter.DbType.In(DbType.String, DbType.AnsiString, DbType.StringFixedLength, DbType.AnsiStringFixedLength))
{
if (parameter.Value == DBNull.Value)
continue;
var value = (string) parameter.Value;
parameter.Size = 4096;
if (value.IndexOf(FullTextPrefix) >= 0)
{
value = value.Replace(FullTextPrefix, ""); // remove prefix we added n linq query
value = value.Substring(1, value.Length-2); // remove %% escaping by linq translator from string.Contains to sql LIKE
parameter.Value = value;
args.Command.CommandText = Regex.Replace(text,
string.Format(#"\(\[(\w*)\].\[(\w*)\]\s*LIKE\s*#{0}\s?(?:ESCAPE '~')\)", parameter.ParameterName),
string.Format(#"contains([$1].[$2], #{0})", parameter.ParameterName));
}
}
}
}
}
And then use it like this:
var fullTextSearch = Db.FullTextPrefix + textToSearch;
var q = Db.Notes.Where(n => !n.Private && n.NoteText.Contains(fullTextSearch));

Answer 4

A slighty nicer method (takes rank into effect) using CONTAINSTABLE
String pkey = context.Mapping.GetTable(typeof(TSource)).RowType.DataMembers.SingleOrDefault(x => x.IsPrimaryKey).Name;
string query = String.Concat(#"SELECT *
FROM ", table, #" AS FT_TBL INNER JOIN
CONTAINSTABLE(", table, #", *, {0}) AS KEY_TBL
ON FT_TBL.", pkey, #" = KEY_TBL.[KEY]
ORDER BY KEY_TBL.[RANK] DESC");
return context.ExecuteQuery<TSource>(query, text);

Answer 5

.NET Core 2.1 and above supports an extension method that allows the use of FREETEXT and FREETEXTTABLE searches
using Microsoft.EntityFrameworkCore;
var results = dbContext.MyTable
.Where(e => EF.Functions.FreeText("*", "search criteria"));
The FreeText function is defined in Microsoft.EntityFrameworkCore.SqlServer so your project must reference that package.
Documentation

Answer 6

I've been trying to solve the exact problem. I like to write my SQL logic in my LINQtoSQL but I needed a way to do Full Text Search. right now I'm just using SQL functions and then calling the user-defined functions inline of the linq queries. not sure if that's the most efficient way. what do you guys think?

Answer 7

You can just do something like this
var results = (from tags in _dataContext.View_GetDeterminationTags
where tags.TagName.Contains(TagName) ||
SqlMethods.Like(tags.TagName,TagName)
select new DeterminationTags
{
Row = tags.Row,
Record = tags.Record,
TagID = tags.TagID,
TagName = tags.TagName,
DateTagged = tags.DateTagged,
DeterminationID = tags.DeterminationID,
DeterminationMemberID = tags.DeterminationMemberID,
MemberID = tags.MemberID,
TotalTagged = tags.TotalTagged.Value
}).ToList();
Notice where TagName.Contains also the SQLMethods.Like just do a using
using System.Data.Linq.SqlClient;
to gain access to that SQLMethods.

Answer 8

dswatik - the reason for wanting full text search is that .contains translates to
SELECT * FROM MYTABLE WHERE COLUMNNAME LIKE '%TEXT%'
Which ignores any indexes and is horrible on a large table.