Using play 2.x, I have the following Form mapping:
val relocationDtoForm: Form[RelocationDto] = Form(
mapping(
"_type" -> text,
"sourceToken" -> text,
"exchange" -> optional(jodaDate("dd/MM/yyyy")),
"completion" -> optional(jodaDate("dd/MM/yyyy")),
"expectedMoveIn" -> optional(jodaDate("dd/MM/yyyy"))
)(RelocationDto.apply)(RelocationDto.unapply)
)
I would like to add validation so that if _type=="sale", then exchange is a required field, but if _type=="let" then expectedMoveIn is a required field. I cant seem to find a way to do this with the standard play validators, is there a way to do it?
Cheers
Nic
I used to work for Her Majesty's Revenue and Customs (UK) and looks like HMRC have open-sourced a very useful and easy to use library specialising in conditional Play mappings: https://github.com/hmrc/play-conditional-form-mapping
You can use the verifying method of Mapping. This will allow you to create constraints after the form has been successfully bound to a class, where you can access other fields. It accepts the error message and a boolean function, which will add an error to the form when false.
val relocationDtoForm: Form[RelocationDto] = Form(
mapping(
"_type" -> text,
"sourceToken" -> text,
"exchange" -> optional(jodaDate("dd/MM/yyyy")),
"completion" -> optional(jodaDate("dd/MM/yyyy")),
"expectedMoveIn" -> optional(jodaDate("dd/MM/yyyy"))
)(RelocationDto.apply)(RelocationDto.unapply)
.verifying(
"Exchange is required for sales.",
relocationRto => if(relocationRto._type == "sale") relocationRto.isDefined else true
).verifying(
"Expected move in is required for let?",
relocationRto => if(relocationRto._type == "let") expectedMoveIn.isDefined else true
)
)
My final solution at long last was:
def save(id: String) = Action { implicit request =>
//Runs some extra validation based on the _type value
def typeSpecificValidate(params: Map[String,Seq[String]]): Seq[FormError] = params("_type") match {
case Seq("sale") => {
Forms.tuple(
"exchange" -> jodaDate("dd/MM/yyyy"),
"completion" -> jodaDate("dd/MM/yyyy")
).bind(params.mapValues(seq => seq.head)) match {
case Left(errors) => errors //Bind returns a Left(List(FormErrors)) or a Right(boundTuple)
case _ => Nil
}
}
case Seq("let") => {
Forms.tuple(
"expectedMoveIn" -> jodaDate("dd/MM/yyyy")
).bind(params.mapValues(seq => seq.head)) match {
case Left(errors) => errors //Bind returns a Left(List(FormErrors)) or a Right(boundTuple)
case _ => Nil
}
}
}
val extraErrors = typeSpecificValidate(request.body.asFormUrlEncoded.get)
Logger.debug("Custom validator found: " + extraErrors)
val ff = relocationDtoForm.bindFromRequest
ff.copy(errors = ff.errors ++ extraErrors).fold(
formWithErrors => {
formWithErrors.errors.foreach(e => Logger.debug(e.toString))
BadRequest(views.html.relocations.detail(id, formWithErrors))
},
relocationDto => {
Logger.debug(relocationDto.toString)
Ok(views.html.relocations.detail(id, relocationDtoForm.fill(relocationDto)))
}
)
}
Related
I have some quite simple .NET logic that I'm transplanting into a Scala codebase, and I don't really know the first thing about Scala. It includes a LINQ query that groups a collection of tagged objects by making use of an anonymous type projection to flatten and join, followed by grouping, eg:
var q = things.SelectMany(t => t.Tags, (t, tag) => new { Thing = t, Tag = tag })
.GroupBy(x => x.Tag, x => x.Thing);
In Scala it looks like flatMap might be of use, but I can't figure out how to combine it with groupBy via an anonymous.
Is this kind of thing a lot more complicated in Scala, or am I missing something simple?
UPDATE:
I ended up going with:
things.flatMap(t => t.Tags.map(x => (x,t))).groupBy(x => x._1)
and then of course later on when I access a value in the map I need to do:
.map(x => x._2)
to get the groups out of the tuple.
Simple when you know how!
Seems to me you want to do something like.
case class Tag(tag:String)
case class Thing(Tags : Seq[Tag])
val things :Seq[Thing] = Seq(Thing(Seq(Tag(""))))
val q = things.map {
thing => new {
val Thing = thing
val Tags = thing.Tags
}
}.flatMap {
thingAndTags => thingAndTags.Tags.map {
tag => new {
val Thing = thingAndTags.Thing
val Tag = tag
}
}
}. groupBy {
thingAndTag => thingAndTag.Tag
}.map {
tagAndSeqOfThingAndTags =>
tagAndSeqOfThingAndTags._1 -> tagAndSeqOfThingAndTags._2.map(x => x.Thing)
}
But in Scala anonymous objects are not really common but you can use Tuple2[T1,T2] instead of all the new { val ...}s,
val q = things.map {
thing => ( thing->thing.Tags)
}.flatMap {
thingAndTags => thingAndTags._2.map {
tag => (thingAndTags._1, tag)
}
}.groupBy {
thingAndTag => thingAndTag._2
}.map {
tagAndSeqOfThingAndTags =>
tagAndSeqOfThingAndTags._1 -> tagAndSeqOfThingAndTags._2.map(x => x._1)
}
its just a little confusing with all the ._1s and ._2s
I have three main questions I need help answering.
How do you correctly map and store a nested map?
How do you search a nested part of a document?
How do you bulk index?
I'm using Nest version 2 and have been looking over the new documentation which can be found Here. The documentation has been useful in creating certain parts of the code but unfortunately doesn't explain how they fit together.
Here is the class I'm trying to map.
[ElasticsearchType(Name = "elasticsearchproduct", IdProperty = "ID")]
public class esProduct
{
public int ID { get; set; }
[Nested]
public List<PriceList> PriceList { get; set; }
}
[ElasticsearchType(Name = "PriceList")]
public class PriceList
{
public int ID { get; set; }
public decimal Price { get; set; }
}
and my mapping code
var node = new Uri(HOST);
var settings = new ConnectionSettings(node).DefaultIndex("my-application");
var client = new ElasticClient(settings);
var map = new CreateIndexDescriptor("my-application")
.Mappings(ms => ms
.Map<esProduct>(m => m
.AutoMap()
.Properties(ps => ps
.Nested<PriceList>(n => n
.Name(c => c.PriceList)
.AutoMap()
)
)
)
);
var response = client.Index(map);
This is the response I get:
Valid NEST response built from a succesful low level call on POST: /my-application/createindexdescriptor
So that seems to work. next index.
foreach (DataRow dr in ProductTest.Tables[0].Rows)
{
int id = Convert.ToInt32(dr["ID"].ToString());
List<PriceList> PriceList = new List<PriceList>();
DataRow[] resultPrice = ProductPriceTest.Tables[0].Select("ID = " + id);
foreach (DataRow drPrice in resultPrice)
{
PriceList.Add(new PriceList
{
ID = Convert.ToInt32(drPrice["ID"].ToString()),
Price = Convert.ToDecimal(drPrice["Price"].ToString())
}
esProduct product = new esProduct
{
ProductDetailID = id,
PriceList = PriceList
};
var updateResponse = client.Update<esProduct>(DocumentPath<esProduct>.Id(id), descriptor => descriptor
.Doc(product)
.RetryOnConflict(3)
.Refresh()
);
var index = client.Index(product);
}
}
Again this seems to work but when I come to search it does seem to work as expected.
var searchResults = client.Search<esProduct>(s => s
.From(0)
.Size(10)
.Query(q => q
.Nested(n => n
.Path(p => p.PriceList)
.Query(qq => qq
.Term(t => t.PriceList.First().Price, 100)
)
)
));
It does return results but I was expecting
.Term(t => t.PriceList.First().Price, 100)
to look move like
.Term(t => t.Price, 100)
and know that is was searching the nested PriceList class, is this not the case?
In the new version 2 documentation I can't find the bulk index section. I tried using this code
var descriptor = new BulkDescriptor();
***Inside foreach loop***
descriptor.Index<esProduct>(op => op
.Document(product)
.Id(id)
);
***Outside foreach loop***
var result = client.Bulk(descriptor);
which does return a success response but when I search I get no results.
Any help would be appreciated.
UPDATE
After a bit more investigation on #Russ advise I think the error must be with my bulk indexing of a class with a nested object.
When I use
var index = client.Index(product);
to index each product I can use
var searchResults = client.Search<esProduct>(s => s
.From(0)
.Size(10)
.Query(q => q
.Nested(n => n
.Path(p => p.PriceList)
.Query(qq => qq
.Term(t => t.PriceList.First().Price, 100)
)
)
)
);
to search and return results, but when I bulk index this no long works but
var searchResults = client.Search<esProduct>(s => s
.From(0)
.Size(10)
.Query(q => q
.Term(t => t.PriceList.First().Price, 100)
)
);
will work, code b doesn't work on the individual index method. Does anyone know why this has happened?
UPDATE 2
From #Russ suggested I have taken a look at the mapping.
the code I'm using to index is
var map = new CreateIndexDescriptor(defaultIndex)
.Mappings(ms => ms
.Map<esProduct>(m => m
.AutoMap()
.Properties(ps => ps
.Nested<PriceList>(n => n
.Name(c => c.PriceList)
.AutoMap()
)
)
)
);
var response = client.Index(map);
Which is posting
http://HOST/fresh-application2/createindexdescriptor {"mappings":{"elasticsearchproduct":{"properties":{"ID":{"type":"integer"},"priceList":{"type":"nested","properties":{"ID":{"type":"integer"},"Price":{"type":"double"}}}}}}}
and on the call to http://HOST/fresh-application2/_all/_mapping?pretty I'm getting
{
"fresh-application2" : {
"mappings" : {
"createindexdescriptor" : {
"properties" : {
"mappings" : {
"properties" : {
"elasticsearchproduct" : {
"properties" : {
"properties" : {
"properties" : {
"priceList" : {
"properties" : {
"properties" : {
"properties" : {
"ID" : {
"properties" : {
"type" : {
"type" : "string"
}
}
},
"Price" : {
"properties" : {
"type" : {
"type" : "string"
}
}
}
}
},
"type" : {
"type" : "string"
}
}
},
"ID" : {
"properties" : {
"type" : {
"type" : "string"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
fresh-application2 returned mapping doesn't mention nested type at all, which I'm guessing is the issue.
The mapping my working nested query looks more like this
{
"my-application2" : {
"mappings" : {
"elasticsearchproduct" : {
"properties" : {
"priceList" : {
"type" : "nested",
"properties" : {
"ID" : {
"type" : "integer"
},
"Price" : {
"type" : "double"
}
}
},
"ID" : {
"type" : "integer"
},
}
}
}
}
}
This has the nested type returned. I think the one which isn't returning nested as a type is when I started using .AutoMap() , am I using it correctly?
UPDATE
I have fixed my mapping problem. I have changed my mapping code to
var responseMap = client.Map<esProduct>(ms => ms
.AutoMap()
.Properties(ps => ps
.Nested<PriceList>(n => n
.Name(c => c.PriceList)
.AutoMap()
)
)
);
Whilst you're developing, I would recommend logging out requests and responses to Elasticsearch so you can see what is being sent when using NEST; this'll make it easier to relate to the main Elasticsearch documentation and also ensure that the body of the requests and responses match your expectations (for example, useful for mappings, queries, etc).
The mappings that you have look fine, although you can forgo the attributes since you are using fluent mapping; there's no harm in having them there but they are largely superfluous (the type name for the esProduct is the only part that will apply) in this case because .Properties() will override inferred or attribute based mapping that is applied from calling .AutoMap().
In your indexing part, you update the esProduct and then immediately after that, index the same document again; I'm not sure what the intention is here but the update call looks superfluous to me; the index call will overwrite the document with the given id in the index straight after the update (and will be visible in search results after the refresh interval). The .RetryOnConflict(3) on the update will use optimistic concurrency control to perform the update (which is effectively a get then index operation on the document inside of the cluster, that will try 3 times if the version of the document changes in between the get and index). If you're replacing the whole document with an update i.e. not a partial update then the retry on conflict is not really necessary (and as per previous note, the update call in your example looks unnecssary altogether since the index call is going to overwrite the document with the given id in the index).
The nested query looks correct; You specify the path to the nested type and then the query to a field on the nested type will also include the path.I'll update the NEST nested query usage documentation to better demonstrate.
The bulk call looks fine; you may want to send documents in batches e.g. bulk index 500 documents at a time, if you need to index a lot of documents. How many to send in one bulk call is going to depend on a number of factors including the document size, how it is analyzed, performance of the cluster, so will need to experiment to get a good bulk size call for your circumstances.
I'd check to make sure that you are hitting the right index, that the index contains the count of documents that you expect and find a document that you know has a PriceList.Price of 100 and see what is indexed for it. It might be quicker to do this using Sense while you're getting up an running.
I'm trying to find the best and cleanest way to handle custom errors in scala.
At present I've a common Global error object (my code looks similar to this):
sealed trait AppError {
def message: String
}
object AppError {
/* Cat Error Messages */
case class CatNotFound(id: Long) extends AppError {
val message = Messages("cat.not_found").format(id)
}
case class CatNotForUser(id: Long) extends AppError {
val message = Messages("cat.not_for_user").format(id)
}
}
Every method is returning Either[<SomeValidType>, AppError] (I'm sending error in right side) for example:
def getCatForUser(id: Long, user: User): Either[Boolean, AppError] = {
CatService get(id) match {
case None => CatNotFound(id)
case Some(x) =>
CatService isAccessibleByUser(x, user) match {
case false => CatNotForUser(id)
case true => true
}
}
}
The method which is calling the validation/get part have a lot of nested match like this:
def something(id: Long, user: User) = {
getCatForUser(id, user) match {
case Left(b) =>
anotherMethodReturnsEither(id) match {
case Left(x) =>
thereCanBeLotOfNesting(user) match {
case Left(y) =>
// do something or create another nesting call
case Right(e) =>
handleSomeHow(e)
}
case Right(e) =>
handleSomeHow(e)
}
case Right(e) =>
handleSomeHow(e)
}
}
def handleSomeHow(e: AppError) = {
e match {
case CatNotFound(_) => NotFound(e.message)
case CatNotForUser(_) => Unauthorized(e.message)
}
}
In one project I was using CustomExceptions to handle such nesting/if else. In that case I was throwing errors which were propagated to the top where they can be handled together. The code looks clean when we are using exceptions instead of errors. But I think its a problem with exceptions when you are validating data & have a method like this that doesn't return anything but throw Exception:
def validateCatForUser(id: Long, user: User) = {
CatService get(id) match {
case None => throw CatNotFound(id)
case Some(x) =>
CatService isAccessibleByUser(x, user) match {
case false => throw CatNotForUser(id)
case true => // nothing can be returned as its validation
}
}
}
and I'll use it like:
def something(id: Long, user: User) = {
validateCatForUser(id, user)
/*
* here will be some logic after validation
*/
}
Still its clean and readable. So my question is what should I use to achieve a more readable code with less match clause having Left() or Right(). I was reading about scala validation and found http://blog.lunatech.com/2012/03/02/validation-scala but scala 'Validation' also returns ValidationNEL[String, <some valid type>] which doesn't reduce match clause. The best way to handle custom errors, in my situation, according to you would be?
for comprehensions really help with the readability/nesting issue when working with Either. You can use left and right to project the Eithers and yield an Either that will collect the error:
val x = Left(5) : Either[Int, String]
val y = Right("error") : Either[Int, String]
val z = Right("error") : Either[Int, String]
for {
xval <- x.left
yval <- y.left
zval <- z.left
} yield zval
Your nested version could be made much cleaner using this syntax. It might look something like this:
def something(id: Long, user: User) = {
for {
cat <- getCatForUser(id, user).left
otherValues <- anotherMethodReturnsEither(id).left
noNesting <- thereCanBeLotOfNesting(user).left
} yield noNesting
} match {
case Left(value) => // do something
case Right(e) => handleSomeHow(e)
}
You should switch to Scalaz's \/ (I call it 'Scalaz Either', or 'Or') and move your 'happy path' to the right side.
\/ is right-side biased which allows you to use it in for-comprehensions. You can write:
for {
resA <- computeA()
resB <- computeB(resA)
} yield resB
where computeX return \/[Error, X].
Following http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters-completion.html
How can I index/insert (I can do mapping) an object using Nest client library to be able to provide following options:
"input": ...,
"output": ...,
"payload" : ...,
"weight" : ...
I would like to be able to provide multiple values in 'input' option.
Can't find anyway of doing this using NEST.
Thank you
NEST provides the SuggestField type in order to assist in indexing completion suggestions. You don't necessarily need to use this type, you can provide your own that contains the expected completion fields (input, output, etc...), but the purpose of SuggestField is to make the whole process easier by already providing a baked in type.
Usage:
Add a suggest field to the document/type you are indexing:
public class MyType
{
public SuggestField Suggest { get; set; }
}
Your mapping should look something like:
client.Map<MyType>(m => m
.Properties(ps => ps
.Completion(c => c.Name(x => x.Suggest).Payloads(true))
)
);
Indexing example:
var myType = new MyType
{
Suggest = new SuggestField
{
Input = new [] { "Nevermind", "Nirvana" },
Output = "Nirvana - Nevermind",
Payload = new { id = 1234 },
Weight = 34
}
};
client.Index<MyType>(myType);
Hope that helps.
case class Address(
address1: String,
city: String,
state: String,
postal: String,
country: String
)
Form(
mapping = mapping(
"address1" -> nonEmptyText,
"city" -> nonEmptyText,
"state" -> nonEmptyText,
"postal" -> nonEmptyText,
"country" -> nonEmptyText
)(Address.apply)(Address.unapply).verifying("Invalid Postal Code!", validatePostal _)
)
def validatePostal(address: Address): Boolean = {
address.country match {
case "US" | "CA" =>
val regex: Regex = ("^(\\d{5}-\\d{4}|\\d{5}|\\d{9})$|^([a-zA-Z]\\d[a-zA-Z]( )?\\d[a-zA-Z]\\d)$").r
regex.pattern.matcher(address.postal).matches()
case _ => false
}
}
Above form validation for postal code works fine with global error displayed on form for invalid US or Canadian postal codes.
I would like to show the error as field error right next to the field than as global error which in my case is shown on top of the form.
Is there a way to use inbuilt Form constraints or verification methods to achieve this instead of FormError's?
You can add the constraint to the field. Then update validatePostal to accept a tuple of the two values instead.
Form(
mapping = mapping(
"address1" -> nonEmptyText,
"city" -> nonEmptyText,
"state" -> nonEmptyText,
"postal" -> tuple(
"code" -> nonEmptyText,
"country" -> nonEmptyText
).verifying("Invalid Postal Code!", validatePostal _),
)((address1, city, state, postal) => Address(address1, city, state, postal._1, postal._2))((address: Address) => Some((address.address1, address.city, address.state, (address.postal, address.country))))
)
Template:
#inputText(
addressForm("postal.code"),
'_label -> "Postal code",
'_help -> "Please enter a valid postal code.",
'_error -> addressForm.error("postal")
)
Defining the error like that you are creating a FormError("","Invalid Postal Code!") object in a form, as it doesn't have key (the first parameter), the framework don't attach the error to the form element.
On form error when you are binding the request to the form, you will have to create a new form removing the FormError("","Invalid Postal Code!") and replacing it with an error FormError("form.id","message")
In our project we created an implicit def for Form to replace the form errors (we couldn't find a way to create dynamic constraint validations) these are the 2 definitions we have:
def replaceError(key: String, newError: FormError): Form[T] = {
val updatedFormErrors = form.errors.flatMap { fe =>
if (fe.key == key) {
if (form.error(newError.key).isDefined) None
else {
if (newError.args.isEmpty ) Some(FormError(newError.key,newError.message,fe.args))
else Some(newError)
}
} else {
Some(fe)
}
}
form.copy(errors = updatedFormErrors.foldLeft(Seq[FormError]()) { (z, fe) =>
if (z.groupBy(_.key).contains(fe.key)) z else z :+ fe
})
}
def replaceError(key: String, message: String, newError: FormError): Form[T] = {
def matchingError(e: FormError) = e.key == key && e.message == message
val oldError = form.errors.find(matchingError)
if (oldError.isDefined) {
val error = if (newError.args.isEmpty) FormError(newError.key,newError.message,oldError.get.args) else newError
form.copy(errors = form.errors.filterNot(e => e.key == key && e.message == message)).withError(error)
}
else form
}
We have those in a class called FormCryptBind (because we also improve the form object with some crypto stuff) and we define the implicit def like this:
implicit def formBinding[T](form: Form[T])(implicit request: Request[_]) = new FormCryptBind[T](form)
We do it like that because just importing the object having this implicit definition, you can use all the FormCryptBind definitions as they were Form's
And we use it like this
import whatever.FormImprovements._
...
object SomeController extends Controller{
...
def submit = Action{ implicit request =>
form.bindRequest.fold(
formWithErrors => {
val newForm = formWithErrors.replaceError("", "formField.required", FormError("formField", "error.required")
BadRequest(someView(newForm)
},
formDetails => Redirect(anotherView(formDetails))
}
As I can't put actual live code from the app, I touched it a little bit :D so expect compile errors if you copy & paste