spark sortByKey key is type of Case Class
rdd.filter(line => {
if(StringUtils.isEmpty(line)){
false
}else{
true
}
}).map(line => {
val array = line.split(",")
(OrderedKey(array(0),array(1)),array(2))
}).repartition(1).sortByKey(true).foreach(println(_))
case class OrderedKey(k1:String,k2:String)
but the result not sort ! why ?
You need to provide an ordering under which your case class instances can be compared. The sortByKey() transformation will then use this ordering to sort your OrderedKey keys.
The following is an example of an ordering in the order of the parameters to the case class:
case class OrderedKey(k1: String, k2: String) extends Ordered[OrderedKey] {
import scala.math.Ordered.orderingToOrdered
def compare(that: OrderedKey): Int = (this.k1, this.k2) compare (that.k1, that.k2)
}
Related
I want to cache a result of a method only when the attribute of the result contains specific values. For example
Class APIOutput(code: Int, message: String)
sealed class Response<out T : Any> : Serializable {
data class Success<out T : Any>(val data: T) : Response<T>()
data class Error(val errorText: String, val errorCode: Int) : Response<Nothing>()
}
#Cacheable(
key = "api-key",
unless = "do something here"
)
fun doApicall(uniqueId: Long): Response<APIOutput> {
//make API call
val output = callAPI(uniqueId)
return Response.Success(output)
}
In the above method, I want to cache the response only when Response.Success.data.code == (long list of codes).
Please note, in the previous line data is nothing but APIOutput object. How could I achieve it using unless or any other approach. I was thinking of writing a function that takes a doApicall method result as input and would return true or false and call that method it as unless="call a method". But I'm not sure how to do it. Any help is highly appreciated.
You can specify an expression to be evaluated in unless using SpEL. The returned value is available as result so you can do something like -
#Cacheable(
key = "api-key",
unless = "#result!=null or #result.success.data.code!=200"
)
fun doApicall(uniqueId: Long): Response<APIOutput> {
//make API call
val output = callAPI(uniqueId)
return Response.Success(output)
}
You can even use Regex in SpEL and can create custom Expression parsers if the existing functionality is not enough for your usecase.
Thanks Yatharth and John! Below is the condition that worked for me. resultcodes in the below expression is a list
#Cacheable(
key = "api-key",
unless = "!(#result instanceof T(com.abc.Response\$Success))
or (#result instanceof T(com.abc.Response\$Success)
and !(T(com.abc.APIStatus).resultCodes.contains(#result.data.code)))"
)
fun doApicall(uniqueId: Long): Response<APIOutput> {
//make API call
val output = callAPI(uniqueId)
return Response.Success(output)
}
I do have a enum class in kotlin in which I have to sort them differently based on a input parameters.
As of now, I have this class defined
enum class ProductItem constructor(
val color,
val price,
val position
){
SHOES("red", "$", 2)
CAR("blue", "$$$$$", 1)
BOAT("green", "$$$$$$$$$$$$$", 3)
}
As of now using a simplify code like this:
it.sortedBy { it.ProductItem?.position })
I was expecting that the list of enum returned will be sorted in ascending order and show: CAR, SHOES and BOAT but it still shows SHOES, CAR, BOAT. It looks like the params position is not took into account. Any idea how to use the params position to sort the list of enum
Thanks
From the look of your code I suspect you have some class (e.g. Order) which has an optional property called ProductItem, which returns a ProductItem enum value. And you then have a list or array of these Order objects, which you want to sort by the position of the associated ProductItem. Is that correct?
If so, here's some code below which shows how the list of orders can be sorted...
enum class ProductItem constructor(
val color: String,
val price: String,
val position: Int
) {
SHOES("red", "$", 2),
CAR("blue", "$$$$$", 1),
BOAT("green", "$$$$$$$$$$$$$", 3)
}
data class Order(val ProductItem: ProductItem? = null)
val orders = listOf(Order(ProductItem.SHOES), Order(ProductItem.CAR), Order(ProductItem.BOAT))
val sortedOrders = orders.sortedBy { it.ProductItem?.position }
Does that give you what you want?
I want to access the keys for an item in enum class
enum class Events {
REFER_AND_EARN {
val key: String = "Refer and Earn"
val source: String = "Source"
},
REFILL_PAST_MEDICINE_CLICK {
val key: String = "Refill Past Medicine Click"
val source: String = "Source"
val pointOfInitiation: String = "Point of initiation"
}
}
Like for the above enum class can I access source like this??
Events.REFER_AND_EARN.source
You can do what you want to achieve by writing this:
enum class Events(val key: String, val source: String, val pointOfInitiation: String? = null) {
REFER_AND_EARN(key = "Refer and Earn", source = "Source"),
REFILL_PAST_MEDICINE_CLICK(
key = "Refill Past Medicine Click",
source = "Source",
pointOfInitiation = "Point of initiation"
)
}
Enum constants do not declare new types themselves. This means that you can't simply access these properties: they are public, but there's no access to the type where they are declared.
You can implement an interface by enum and expose these properties by overriding ones from interface.
Or you can declare a sealed class instead of enum class and use object declarations instead of enum constants.
You need to use properties instead:
enum class Events(val key: String,
val source: String,
val pointOfInitiation: String) {
REFER_AND_EARN("Refer and Earn",
"Source",
"Unknown"),
REFILL_PAST_MEDICINE_CLICK(
"Refill Past Medicine Click",
"Source",
"Point of initiation"
);
}
Or you can use a sealed class as others mentioned.
We have a class like this in a Grails 2.4.3 application (migrated from 2.3.8):
#Validateable
class Foo {
Integer noDefault;
Integer withDefault = 1;
static constraints = {
noDefault(nullable:false)
withDefault(nullable:false)
}
}
This class is being instantiated in a complex configuration mechanism using a Map like this:
[
noDefault: 0,
withDefault: 2
]
(In fact the Map is part of a huge one, but the class constructor sees this small one.) Formerly the class worked if we omitted the withDefault entry from the config map, using the default value which is not null. In Grails 2.4.3, however, it tells me that this field cannot be null. I can fix it by letting it be null in the constraint, but it lets setting the explicite value null (and overwrite the default value), which causes problem during operation.
Do you know some workaround, which preserves the semantics and correct operation?
Thanx in advance, best regards: Balázs
What you are describing is not consistent with what I would expect and not consistent with the behavior I am seeing. The project at https://github.com/jeffbrown/validatedefaults contains the following code.
At https://github.com/jeffbrown/validatedefaults/blob/master/src/groovy/demo/Foo.groovy
// src/groovy/demo/Foo.groovy
package demo
import grails.validation.Validateable
#Validateable
class Foo {
Integer noDefault;
Integer withDefault = 1;
static constraints = {
noDefault(nullable:false)
withDefault(nullable:false)
}
}
The test at https://github.com/jeffbrown/validatedefaults/blob/master/test/unit/demo/FooSpec.groovy passes:
// test/unit/demo/FooSpec.groovy
package demo
import spock.lang.Specification
import grails.test.mixin.TestMixin
import grails.test.mixin.support.GrailsUnitTestMixin
#TestMixin(GrailsUnitTestMixin)
class FooSpec extends Specification {
void 'test validating default values'() {
given:
def map = [noDefault: 0]
def foo = new Foo(map)
expect:
foo.validate()
}
}
When I run the app I get the same behavior.
// grails-app/conf/BootStrap.groovy
import demo.Foo
class BootStrap {
def init = { servletContext ->
def map = [noDefault: 0]
def foo = new Foo(map)
// this prints true...
println "Foo is valid? : ${foo.validate()}"
}
def destroy = {
}
}
I hope that helps.
I'm using Constraints on my web forms and I've noticed that several forms have similar validations, for instance I have several types of form with a start date and an end date. In each case, I want to validate that the start date is before the end date. Here's the case class I'm creating from my form:
case class OrderSearchForm(orderId: Option[Int], startDate:Option[Long], endDate:Option[Long])
and my validation (Let's ignore the .get() for now):
def validateSearchDate = Constraint[OrderSearchForm]{
osf: OrderSearchForm => {
if (!osf.startDate.isEmpty && !osf.endDate.isEmpty && osf.startDate.get.compareTo(osf.endDate.get) > 0 )
Invalid("Begin Date is after End Date.")
else
Valid
}
}
Now, since I have lots of forms with a start date and an end date, I'd like to re-write my validation to work with all of the case classes representing these forms. I'm wondering whether the typeclass pattern can help me with this:
trait TwoDates[T] {
def twoDatesTuple(t: T): (Option[Long], Option[Long])
}
trait TwoDatesOSF extends TwoDates[OrderSearchForm] {
def twoDatesTuple(t: OrderSearchForm) = (t.startDate, t.endDate)
}
implicit object TwoDatesOSF extends trait TwoDatesOSF
def validateSearchDate = Constraint[TwoDates[_]] { t: TwoDates[_] => ... (as above)}
but applying does not work:
validateSearchDate(OrderSearchForm(None, None, None))
yields:
error: type mismatch; found : OrderSearchForm required:
TwoDates[_]
betweenDates(osf)
1) Can I write generic validations using typeclasses? If so, what am I doing wrong?
2) Can I write generic validations while AVOIDING using super-classes (i.e.
abstract class TwoDates(start: Option[Long], end:Option[Long])
case class OrderSearchForm(orderId: Option[String], startDate:Option[Long], endDate:Option[Long]) extends TwoDates(startDate, endDate)
which seems awkward once multiple validations are in play)
Thanks!
I think you can use structural types:
private type TwoDates = { def startDate: Option[Date]; def endDate: Option[Date] }
def validateTwoDates = Constraint[TwoDates] { osf: TwoDates =>
if (!osf.startDate.isEmpty &&
!osf.endDate.isEmpty &&
osf.startDate.get.compareTo(osf.endDate.get) > 0) {
Invalid("Begin Date is after End Date.")
} else Valid
}
case class Something(
startDate: Option[Date],
endDate: Option[Date],
name: String)
private val form = Form(mapping(
"startDate" -> optional(date),
"endDate" -> optional(date),
"name" -> text)
(Something.apply)(Something.unapply).verifying(validateTwoDates))