How can I get the time it takes a function to test the performance of functions in Kotlin - performance

I need to check how long does a function need to run. I have the following functions which address the same task:
mixAnimalsA
fun mixAnimalsA(a1: Animal, a2: Animal) =
when (setOf(a1, a2)) {
setOf(Animal.OWL, Animal.Leopard) -> Beast.OWLPARD
setOf(Animal.ELEPHANT, Animal.BUTTERFLY) -> Beast.BUTTERPHANT
else -> throw Exception("Not possible combination")
}
mixAnimalsB
fun mixAnimalsB(a1: Animal, a2: Animal) =
when (setOf(a1, a2)) {
(c1 == Animal.OWL && c2 == Animal.Leopard) ||
(c2 == Animal.OWL && c1 == Animal.Leopard) -> Beast.OWLPARD
(c1 == Animal.ELEPHANT && c2 == Animal.BUTTERFLY) ||
(c2 == Animal.ELEPHANT && c1 == Animal.BUTTERFLY)-> Beast.BUTTERPHANT
else -> throw Exception("Not possible combination")
}
Animal and Beast are enumerations. How can I measure how long each function takes to run?

If you're looking for an in-code solution, you can use measureTimeMillis and measureNanoTime, like this:
val time = measureTimeMillis {
// call your function here
}
They return the measured time in milliseconds and nanoseconds, respectively.

Measure execution time and also keep the result
Standard Library
The standard library function measureTimedValue may be used to measure execution time and at the same time capture the result. This tuple of values is being exposed as a TimedValue(value: T, duration: Duration):
#ExperimentalTime
fun main() {
val (result: String, duration: Duration) = measureTimedValue {
operation()
}
print("Got $result after ${duration.inMilliseconds} ms.")
}
Note that this API is experimental and requires explicit opt-in.
Obsolete custom implementation
(This used to be my answer before the standard lib was extended)
If you want to measure the execution time and also access the measured function's return value afterward, here's a custom solution:
inline fun <R> executeAndMeasureTimeMillis(block: () -> R): Pair<R, Long> {
val start = System.currentTimeMillis()
val result = block()
return result to (System.currentTimeMillis() - start)
}
You can call it like this:
val (response, duration) = executeAndMeasureTimeMillis {
restTemplate.getForObject<AnyObject>(uri)
}

If it's enough to get the time as output on the console:
fun <T> timeIt(message: String = "", block: () -> T): T {
val start = System.currentTimeMillis()
val r = block()
val end = System.currentTimeMillis()
println("$message: ${end - start} ms")
return r
}
Usage:
val result = timeIt("note about the code") {
// do something...
}
Output (example):
note about the code: 1ms

For the benchmark of some code block and getting the result a same time, i do some refactor of the standard method in TimingKt class
to give us output generic result and at the same time display a given log.
Here is my example :
/**
* Executes the given block and show the elapsed time in milliseconds in a given message.
*
* #param block which will be bench marked
* #param logMessage log message to be displayed
*
* #return a generic result
*
*/
private fun <T> measureTime(block: () -> T, logMessage: String): T {
val start = System.currentTimeMillis()
val t = block()
val consumedTime = System.currentTimeMillis() - start
Log.d(TAG, "Calculation of $logMessage time :$consumedTime milliseconds")
return t
}
And how it will be used :
return measureTime({
// given block with return result
}, "message to be displayed typically the name of method which will be calculated")

This is my simple time test code.
class TimeCounter(val name: String) {
var totalTime: Long = 0
private set
var count: Int = 0
private set
var minTime: Long = Long.MAX_VALUE
private set
var maxTime: Long = Long.MIN_VALUE
private set
fun addTime(time: Long) {
this.count++
this.totalTime += time
if (this.minTime > time) {
this.minTime = time
}
if (this.maxTime < time) {
this.maxTime = time
}
}
val averageTime: Double
get() = this.totalTime / this.count.toDouble()
fun printTime() {
println("(time about : '$name'), totalTime : $totalTime, count : $count, " +
"average : $averageTime, minTime : $minTime, maxTime : $maxTime")
}
fun <T> runWithTimeCount(run: () -> T): T {
val startTime = System.currentTimeMillis()
return run().also {
this.addTime(System.currentTimeMillis() - startTime)
}
}
}
you can use 'TimeCounter' like this, (example)
var sum = 0
val testTimeCounter = TimeCounter("logic1")
for(i in 0 until 100){
sum += testTimeCounter.runWithTimeCount {
logic1(i) // your logic
}
}
println(sum)
testTimeCounter.printTime() // totalTime, average, minTime, maxTime

Execute function, measure its performance and logs performance - in same call
this solution will help folks who want to measure and log performance, execute function at same time, also is a cleaner approach when there is multiple performance measurement involved of different functions
Create functions as such:
//the inline performance measurement method
private inline fun <T> measurePerformanceInMS(
logger: (Long) -> Unit,
function: () -> T)
: T {
val startTime = System.currentTimeMillis()
val result: T = function.invoke()
val endTime = System.currentTimeMillis()
logger.invoke( endTime - startTime)
return result
}
//the logger function
fun logPerf(time: Long){
Log.d("TAG","PERFORMANCE IN MS: $time ms ")
}
//the function whose performance needs to be checked
fun longRunningFunction() : Int{
var x = 0
for (i in 1..20000) x++
return x
}
This way you can keep logging, performance computation and function execution under a single function call and no code replication needed.
If you require nano second measurement then use System.nanoTime()
USAGE :
val endResult = measurePerformanceInMS({time -> logPerf(time)}){
longRunningFunction()
}
NOTE : here the 'endResult' will carry the result of function whose performance was being measured.

Related

How to write expression in java8?

I'm trying to write this function with java 8 , but i didn't succeeded.
This is my code :
private LocalTime getAbsenceParJour(List<test> tests) {
LocalTime absenceParJourTime = null;
Duration sum = Duration.ZERO;
for (test perPrestation : tests) {
if (perPrestation.getTypeContactLibre().equalsIgnoreCase("absence")) {
Duration duration = Duration.between(perPrestation.getHeureDebut(), perPrestation.getHeureFin());
sum = sum.plus(duration);
}
}
if (sum != null) {
long m = sum.toMinutes() % 60;
long s = sum.toHours() % 60;
absenceParJourTime = LocalTime.of(Long.valueOf(s).intValue(), Long.valueOf(m).intValue());
}
return absenceParJourTime;
}
where test is a class java this is :
public class Prestation {
private LocalTime heureDebut;
private LocalTime heureFin;
private String typeContactLibre;
}
Any help thanks in advance.
private LocalTime getAbsenceParJour(List<test> tests) {
Duration sum = tests.stream()
.filter(P-> P.getTypeContactLibre().equalsIgnoreCase("absence"))
.reduce(null, (sum, p) ->
(sum == null ? Duration.ZERO : sum)
.plus(Duration.between(p.getHeureDebut(), p.getHeureFin()));
LocalTime absenceParJourTime = null;
if (sum != null) {
int m = (int)(sum.toMinutes() % 60);
int s = (int)(sum.toHours() % 60);
absenceParJourTime = LocalTime.of(s, m);
}
return absenceParJourTime;
}
Use of Stream is tricky here:
There is a .sum but not for Duration. So you must use reduce.
You want to distinguish between a Duration.ZERO found, and nothing found.
For durations I think you could do without the use of null.
I hope you're asking to replace your foreach with java8 then below is syntax :
tests.stream().forEach(tests -> {
if (perPrestation.getTypeContactLibre().equalsIgnoreCase("absence")) {
Duration duration = Duration.between(perPrestation.getHeureDebut(), perPrestation.getHeureFin());
sum = sum.plus(duration);
}
});

Replacing for loops for searching list in kotlin

I am trying to convert my code as clean as possible using the Kotlin's built-in functions. I have done some part of the code using for loops. But I want to know the efficient built-in functions to be used for this application
I have two array lists accounts and cards.
My goal is to search a specific card with the help of its card-number, in the array list named cards.
Then I have to validate the pin. If the pin is correct, by getting that gift card's customerId I have to search the account in the array list named accounts. Then I have to update the balance of the account.
These are the class which I have used
class Account{
constructor( )
var id : String = generateAccountNumber()
var name: String? = null
set(name) = if (name != null) field = name.toUpperCase() else { field = "Unknown User"; println("invalid details\nAccount is not Created");}
var balance : Double = 0.0
set(balance) = if (balance >= 0) field = balance else { field = 0.0 }
constructor(id: String = generateAccountNumber(), name: String?,balance: Double) {
this.id = id
this.balance = balance
this.name = name
}
}
class GiftCard {
constructor( )
var cardNumber : String = generateCardNumber()
var pin: String? = null
set(pin) = if (pin != null) field = pin else { field = "Unknown User"; println("Please set the pin\nCard is not Created");}
var customerId : String = ""
set(customerId) = if (customerId != "") field = customerId else { field = "" }
var cardBalance : Double = 0.0
set(cardBalance) = if (cardBalance > 0) field = cardBalance else { field = 0.0; println("Card is created with zero balance\nPlease deposit") }
var status = Status.ACTIVE
constructor(cardNumber: String = generateCardNumber(),
pin: String,
customerId: String,
cardBalance: Double = 0.0,
status: Status = Status.ACTIVE){
this.cardNumber = cardNumber
this.pin = pin
this.customerId = customerId
this.cardBalance = cardBalance
this.status = status
}
}
This is the part of code, I have to be changed :
override fun closeCard(cardNumber: String, pin: String): Pair<Boolean, Boolean> {
for (giftcard in giftcards) {
if (giftcard.cardNumber == cardNumber) {
if (giftcard.pin == pin) {
giftcard.status = Status.CLOSED
for (account in accounts)
account.balance = account.balance + giftcard.cardBalance
giftcard.cardBalance = 0.0
return Pair(true,true)
}
\\invalid pin
return Pair(true,false)
}
}
\\card is not present
return Pair(false,false)
}
Both classes are not very idiomatic. The primary constructor of a Kotlin class is implicit and does not need to be defined, however, you explicitly define a constructor and thus you add another one that is empty.
// good
class C
// bad
class C {
constructor()
}
Going further, Kotlin has named arguments and default values, so make use of them.
class Account(
val id: String = generateAccountNumber(),
val name: String = "Unknown User",
val balance: Double = 0.0
)
Double is a very bad choice for basically anything due to its shortcomings, see for instance https://www.floating-point-gui.de/ Choosing Int, Long, heck even BigDecimal would be better. It also seems that you don’t want the balance to ever go beneath zero, in that case consider UInt and ULong.
Last but not least is the mutability of your class. This can make sense but it also might be dangerous. It is up to you to decide upon your needs and requirements.
enum class Status {
CLOSED
}
#ExperimentalUnsignedTypes
class Account(private var _balance: UInt) {
val balance get() = _balance
operator fun plusAssign(other: UInt) {
_balance += other
}
}
#ExperimentalUnsignedTypes
class GiftCard(
val number: String,
val pin: String,
private var _status: Status,
private var _balance: UInt
) {
val status get() = _status
val balance get() = _balance
fun close() {
_status = Status.CLOSED
_balance = 0u
}
}
#ExperimentalUnsignedTypes
class Main(val accounts: List<Account>, val giftCards: List<GiftCard>) {
fun closeCard(cardNumber: String, pin: String) =
giftCards.find { it.number == cardNumber }?.let {
(it.pin == pin).andAlso {
accounts.forEach { a -> a += it.balance }
it.close()
}
}
}
inline fun Boolean.andAlso(action: () -> Unit): Boolean {
if (this) action()
return this
}
We change the return type from Pair<Boolean, Boolean> to a more idiomatic Boolean? where Null means that we did not find anything (literally the true meaning of Null), false that the PIN did not match, and true that the gift card was closed. We are not creating a pair anymore and thus avoid the additional object allocation.
The Boolean.andAlso() is a handy extension function that I generally keep handy, it is like Any.also() from Kotlin’s STD but only executes the action if the Boolean is actually true.
There's probably a million different ways to do this, but here's one that at least has some language features I feel are worthy to share:
fun closeCard(cardNumber: String, pin: String): Pair<Boolean, Boolean> {
val giftCard = giftcards.find { it.cardNumber == cardNumber }
?: return Pair(false, false)
return if (giftCard.pin == pin) {
giftCard.status = Status.CLOSED
accounts.forEach {
it.balance += giftCard.cardBalance
}
Pair(true, true)
} else
Pair(true, false)
}
The first thing to notice if the Elvis operator - ?: - which evaluates the right side of the expression if the left side is null. In this case, if find returns null, which is equivalent to not finding a card number that matches the desired one, we'll immediately return Pair(false, false). This is the last step in your code.
From there one it's pretty straight forward. If the pins match, you loop through the accounts list with a forEach and close the card. If the pins don't match, then we'll go straight to the else branch. In kotlin, if can be used as an expression, therefore we can simply put the return statement before the if and let it return the result of the last expression on each branch.
PS: I won't say this is more efficient than your way. It's just one way that uses built-in functions - find and forEach - like you asked, as well as other language features.
PPS: I would highly recommend to try and find another way to update the lists without mutating the objects. I don't know your use cases, but this doesn't feel too thread-safe. I didn't post any solution for this, because it's outside the scope of this question.

Using the filter function in kotlin

so the past couple of hours, i have been trying to understand how the filter function works in kotlin and if it has any correlation with that of Java.
basically, i have a code that's written in java and i would love to have it transcribed to kotlin
private List<Order> getFilteredOrders(Courier courier) {
String[] glovoBoxKeywords = glovoBoxWords.toLowerCase().split(",");
List<Vehicle> allowedVehicles = Arrays.asList(MOTORCYCLE, ELECTRIC_SCOOTER);
return orders.stream()
.filter(order -> {
String description = order.getDescription().toLowerCase();
if (!courier.getBox()) {
return Arrays.stream(glovoBoxKeywords).noneMatch(description::contains);
}
return true;
})
.filter(order -> {
Location pickupLocation = order.getPickup();
Location deliveryLocation = order.getDelivery();
Double distance = calculateDistance(pickupLocation, deliveryLocation);
if (distance > longDeliveryDistance) {
return allowedVehicles.contains(courier.getVehicle());
}
return true;
})
.collect(Collectors.toList());
}
i tried this but i got at this, and was literally stuck :(
private fun findFilteredOrder(courier: Courier) : List<Order> {
val glovoBoxKeyWords = glovoBoxWords.toLowerCase().split(",")
val allowedVehicles = listOf(Vehicle.ELECTRIC_SCOOTER, Vehicle.MOTORCYCLE)
orderList.filter { order ->
val description = order.getDescription().toLowerCase()
if(!courier.getBox()) {
}
true
}.filter {
val pickupLocation = it.getPickup()
val deliveryLocation = it.getDelivery()
val distance = calculateDistance(deliveryLocation, pickupLocation)
if(distance > longDeliveryDistance) {
courier.getVehicle() in allowedVehicles
}
true
}
}
Please this is my first attempt and doing something with kotlin, so please go easy guys. thanks, also i'd be appreciative if anyone could help me with informative stuff as to how to understand these kotlin functions better. let, apply, associateBy... etc.. THANKS
The filter function in Kotlin Collections has the same principle as other frameworks/libraries, including Java Streams. Given a predicate (a function from the type of the collection to Boolean) it will return a new collection with the elements matching the predicate. You can find more information and examples of other functions and operators in the official documentation and here.
Your code was almost there, I translate the Java Stream operation to Kotlin List and rewrite the return statements to remove the redundant if
private fun findFilteredOrder(courier: Courier) : List<Order> {
val glovoBoxKeyWords = glovoBoxWords.toLowerCase().split(",")
val allowedVehicles = listOf(Vehicle.ELECTRIC_SCOOTER, Vehicle.MOTORCYCLE)
orderList.filter { order ->
val description = order.getDescription().toLowerCase()
courier.getBox() || glovoBoxKeywords.none { it in description }
}.filter { order ->
val pickupLocation = order.getPickup()
val deliveryLocation = order.getDelivery()
val distance = calculateDistance(deliveryLocation, pickupLocation)
distance <= longDeliveryDistance || courier.getVehicle() in allowedVehicles
}
}
I don't know why no one mentioned the use of labels: https://kotlinlang.org/docs/returns.html#break-and-continue-labels.
Since this question has a nice google ranking, I'll add what I was originally searching for.
The OP probably was aware that filter needs a predicate that returns a Boolean and that the filter will return a list with the items that pass the predicate (the items which the predicate returned true).
What he was not aware is that we can "emulate" Java returns through Kotlin labels:
private fun findFilteredOrder(courier: Courier) : List<Order> {
val glovoBoxKeyWords = glovoBoxWords.toLowerCase().split(",")
val allowedVehicles = listOf(Vehicle.ELECTRIC_SCOOTER, Vehicle.MOTORCYCLE)
orderList.filter shouldSkip#{ order ->
val description = order.getDescription().toLowerCase()
if (courier.getBox()) {
return#shouldSkip true
}
if (glovoBoxKeywords.none { it in description }) {
return#shouldSkip true
}
return#shouldSkip false
}.filter shouldSkip# { order ->
val pickupLocation = order.getPickup()
val deliveryLocation = order.getDelivery()
val distance = calculateDistance(deliveryLocation, pickupLocation)
if (distance <= longDeliveryDistance) {
return#shouldSkip true
}
if (courier.getVehicle() in allowedVehicles) {
return#shouldSkip true
}
return#shouldSkip false
}
}
Since Kotlin allows us to return in the last block line and the return keyword returns to the outer scope, it is pretty easy to:
filter {
startPutting >= someMagic && andComplex ||
verificationsThat.is { hardToUnderstand }.because {
weNeedToReturnHere
}
}
The labels allow us to be more verbose but also more clear.

how the function FutureTask.awaitdone() works?

recently,I read the source code about Java concurrent Jar,the code in FutureTask class is very hard to understand,the awaitDone method like this:
private int awaitDone(boolean timed, long nanos)
throws InterruptedException {
final long deadline = timed ? System.nanoTime() + nanos : 0L;
WaitNode q = null;
boolean queued = false;
for (;;) {
if (Thread.interrupted()) {
removeWaiter(q);
throw new InterruptedException();
}
int s = state;
if (s > COMPLETING) {
if (q != null)
q.thread = null;
return s;
}
else if (s == COMPLETING) // cannot time out yet
Thread.yield();
else if (q == null)
q = new WaitNode();
else if (!queued)
queued = UNSAFE.compareAndSwapObject(this, waitersOffset,
q.next = waiters, q);
else if (timed) {
nanos = deadline - System.nanoTime();
if (nanos <= 0L) {
removeWaiter(q);
return state;
}
LockSupport.parkNanos(this, nanos);
}
else
LockSupport.park(this);
}
}
so,the question is ,I don't understand the usage of waiters, and how is the removeWaiter method worked.
any help would be appreciated.
According to the FutureTask javadocs, the waiters datastructure is a Treiber Stack, a lock-free data structure.
When one thread notices that the future is set or cancelled, it notifies any other waiting threads they can continue (un-parks them).

Determine Event Recurrence Pattern for a set of dates

I am looking for a pattern, algorithm, or library that will take a set of dates and return a description of the recurrence if one exits, i.e. the set [11-01-2010, 11-08-2010, 11-15-2010, 11-22-2010, 11-29-2010] would yield something like "Every Monday in November".
Has anyone seen anything like this before or have any suggestions on the best way to implement it?
Grammatical Evolution (GE) is suitable for this kind of problem, because you are searching for an answer that adheres to a certain language. Grammatical Evolution is also used for program generation, composing music, designing, etcetera.
I'd approach the task like this:
Structure the problem space with a grammar.
Construct a Context-free Grammar that can represent all desired recurrence patterns. Consider production rules like these:
datepattern -> datepattern 'and' datepattern
datepattern -> frequency bounds
frequency -> 'every' ordinal weekday 'of the month'
frequency -> 'every' weekday
ordinal -> ordinal 'and' ordinal
ordinal -> 'first' | 'second' | 'third'
bounds -> 'in the year' year
An example of a pattern generated by these rules is: 'every second and third wednesday of the month in the year 2010 and every tuesday in the year 2011'
One way to implement such a grammar would be through a class hierarchy that you will later operate on through reflection, as I've done in the example below.
Map this language to a set of dates
You should create a function that takes a clause from your language and recursively returns the set of all dates covered by it. This allows you to compare your answers to the input.
Guided by the grammar, search for potential solutions
You could use a Genetic algorithm or Simulated Annealing to match the dates to the grammar, try your luck with Dynamic Programming or start simple with a brute force enumeration of all possible clauses.
Should you go with a Genetic Algorithm, your mutation concept should consist of substituting an expression for another one based on the application of one of your production rules.
Have a look at the following GE-related sites for code and information:
http://www.bangor.ac.uk/~eep201/jge/
http://nohejl.name/age/
http://www.geneticprogramming.us/Home_Page.html
Evaluate each solution
The fitness function could take into account the textual length of the solution, the number of dates generated more than once, the number of dates missed, as well as the number of wrong dates generated.
Example code
By request, and because it's such an interesting challenge, I've written a rudimentary implementation of the algorithm to get you started. Although it works it is by no means finished, the design should definitively get some more thought, and once you have gleaned the fundamental take-aways from this example I recommend you consider using one the libraries I've mentioned above.
/// <summary>
/// This is a very basic example implementation of a grammatical evolution algorithm for formulating a recurrence pattern in a set of dates.
/// It needs significant extensions and optimizations to be useful in a production setting.
/// </summary>
static class Program
{
#region "Class hierarchy that codifies the grammar"
class DatePattern
{
public Frequency frequency;
public Bounds bounds;
public override string ToString() { return "" + frequency + " " + bounds; }
public IEnumerable<DateTime> Dates()
{
return frequency == null ? new DateTime[] { } : frequency.FilterDates(bounds.GetDates());
}
}
abstract class Bounds
{
public abstract IEnumerable<DateTime> GetDates();
}
class YearBounds : Bounds
{
/* in the year .. */
public int year;
public override string ToString() { return "in the year " + year; }
public override IEnumerable<DateTime> GetDates()
{
var firstDayOfYear = new DateTime(year, 1, 1);
return Enumerable.Range(0, new DateTime(year, 12, 31).DayOfYear)
.Select(dayOfYear => firstDayOfYear.AddDays(dayOfYear));
}
}
abstract class Frequency
{
public abstract IEnumerable<DateTime> FilterDates(IEnumerable<DateTime> Dates);
}
class WeeklyFrequency : Frequency
{
/* every .. */
public DayOfWeek dayOfWeek;
public override string ToString() { return "every " + dayOfWeek; }
public override IEnumerable<DateTime> FilterDates(IEnumerable<DateTime> Dates)
{
return Dates.Where(date => (date.DayOfWeek == dayOfWeek));
}
}
class MonthlyFrequency : Frequency
{
/* every .. */
public Ordinal ordinal;
public DayOfWeek dayOfWeek;
/* .. of the month */
public override string ToString() { return "every " + ordinal + " " + dayOfWeek + " of the month"; }
public override IEnumerable<DateTime> FilterDates(IEnumerable<DateTime> Dates)
{
return Dates.Where(date => (date.DayOfWeek == dayOfWeek) && (int)ordinal == (date.Day - 1) / 7);
}
}
enum Ordinal { First, Second, Third, Fourth, Fifth }
#endregion
static Random random = new Random();
const double MUTATION_RATE = 0.3;
static Dictionary<Type, Type[]> subtypes = new Dictionary<Type, Type[]>();
static void Main()
{
// The input signifies the recurrence 'every first thursday of the month in 2010':
var input = new DateTime[] {new DateTime(2010,12,2), new DateTime(2010,11,4),new DateTime(2010,10,7),new DateTime(2010,9,2),
new DateTime(2010,8,5),new DateTime(2010,7,1),new DateTime(2010,6,3),new DateTime(2010,5,6),
new DateTime(2010,4,1),new DateTime(2010,3,4),new DateTime(2010,2,4),new DateTime(2010,1,7) };
for (int cTests = 0; cTests < 20; cTests++)
{
// Initialize with a random population
int treesize = 0;
var population = new DatePattern[] { (DatePattern)Generate(typeof(DatePattern), ref treesize), (DatePattern)Generate(typeof(DatePattern), ref treesize), (DatePattern)Generate(typeof(DatePattern), ref treesize) };
Run(input, new List<DatePattern>(population));
}
}
private static void Run(DateTime[] input, List<DatePattern> population)
{
var strongest = population[0];
int strongestFitness = int.MinValue;
int bestTry = int.MaxValue;
for (int cGenerations = 0; cGenerations < 300 && strongestFitness < -100; cGenerations++)
{
// Select the best individuals to survive:
var survivers = population
.Select(individual => new { Fitness = Fitness(input, individual), individual })
.OrderByDescending(pair => pair.Fitness)
.Take(5)
.Select(pair => pair.individual)
.ToArray();
population.Clear();
// The survivers are the foundation for the next generation:
foreach (var parent in survivers)
{
for (int cChildren = 0; cChildren < 3; cChildren++)
{
int treeSize = 1;
DatePattern child = (DatePattern)Mutate(parent, ref treeSize); // NB: procreation may also be done through crossover.
population.Add((DatePattern)child);
var childFitness = Fitness(input, child);
if (childFitness > strongestFitness)
{
bestTry = cGenerations;
strongestFitness = childFitness;
strongest = child;
}
}
}
}
Trace.WriteLine("Found best match with fitness " + Fitness(input, strongest) + " after " + bestTry + " generations: " + strongest);
}
private static object Mutate(object original, ref int treeSize)
{
treeSize = 0;
object replacement = Construct(original.GetType());
foreach (var field in original.GetType().GetFields())
{
object newFieldValue = field.GetValue(original);
int subtreeSize;
if (field.FieldType.IsEnum)
{
subtreeSize = 1;
if (random.NextDouble() <= MUTATION_RATE)
newFieldValue = ConstructRandomEnumValue(field.FieldType);
}
else if (field.FieldType == typeof(int))
{
subtreeSize = 1;
if (random.NextDouble() <= MUTATION_RATE)
newFieldValue = (random.Next(2) == 0
? Math.Min(int.MaxValue - 1, (int)newFieldValue) + 1
: Math.Max(int.MinValue + 1, (int)newFieldValue) - 1);
}
else
{
subtreeSize = 0;
newFieldValue = Mutate(field.GetValue(original), ref subtreeSize); // mutate pre-maturely to find out subtreeSize
if (random.NextDouble() <= MUTATION_RATE / subtreeSize) // makes high-level nodes mutate less.
{
subtreeSize = 0; // init so we can track the size of the subtree soon to be made.
newFieldValue = Generate(field.FieldType, ref subtreeSize);
}
}
field.SetValue(replacement, newFieldValue);
treeSize += subtreeSize;
}
return replacement;
}
private static object ConstructRandomEnumValue(Type type)
{
var vals = type.GetEnumValues();
return vals.GetValue(random.Next(vals.Length));
}
private static object Construct(Type type)
{
return type.GetConstructor(new Type[] { }).Invoke(new object[] { });
}
private static object Generate(Type type, ref int treesize)
{
if (type.IsEnum)
{
return ConstructRandomEnumValue(type);
}
else if (typeof(int) == type)
{
return random.Next(10) + 2005;
}
else
{
if (type.IsAbstract)
{
// pick one of the concrete subtypes:
var subtypes = GetConcreteSubtypes(type);
type = subtypes[random.Next(subtypes.Length)];
}
object newobj = Construct(type);
foreach (var field in type.GetFields())
{
treesize++;
field.SetValue(newobj, Generate(field.FieldType, ref treesize));
}
return newobj;
}
}
private static int Fitness(DateTime[] input, DatePattern individual)
{
var output = individual.Dates().ToArray();
var avgDateDiff = Math.Abs((output.Average(d => d.Ticks / (24.0 * 60 * 60 * 10000000)) - input.Average(d => d.Ticks / (24.0 * 60 * 60 * 10000000))));
return
-individual.ToString().Length // succinct patterns are preferred.
- input.Except(output).Count() * 300 // Forgetting some of the dates is bad.
- output.Except(input).Count() * 3000 // Spurious dates cause even more confusion to the user.
- (int)(avgDateDiff) * 30000; // The difference in average date is the most important guide.
}
private static Type[] GetConcreteSubtypes(Type supertype)
{
if (subtypes.ContainsKey(supertype))
{
return subtypes[supertype];
}
else
{
var types = AppDomain.CurrentDomain.GetAssemblies().ToList()
.SelectMany(s => s.GetTypes())
.Where(p => supertype.IsAssignableFrom(p) && !p.IsAbstract).ToArray();
subtypes.Add(supertype, types);
return types;
}
}
}
Hope this gets you on track. Be sure to share your actual solution somewhere; I think it will be quite useful in lots of scenarios.
If your purpose is to generate human-readable descriptions of the pattern, as in your "Every Monday in November", then you probably want to start by enumerating the possible descriptions. Descriptions can be broken down into frequency and bounds, for example,
Frequency:
Every day ...
Every other/third/fourth day ...
Weekdays/weekends ...
Every Monday ...
Alternate Mondays ...
The first/second/last Monday ...
...
Bounds:
... in January
... between 25 March and 25 October
...
There won't be all that many of each, and you can check for them one by one.
What I would do:
Create samples of the data
Use a clustering algorithm
Generate samples using the algorithm
Creating a fitness function to measure how well it correlates to the full data set. The clustering algorithm will come up with either 0 or 1 suggestions and you can meassure it against how well it fits in with the full set.
Elementate/merge the occurrence with the already found sets and rerun this algorithm.
Looking at that you may want to use either Simulated Annealing, or an Genetic Algorithm. Also, if you have the descriptions, you may want to compare the descriptions to generate a sample.
You could access the system date or system dateandtime and construct crude calendar points in memory based on the date and the day of the week as returned by the call or function result. Then use the number of days in relevant months to sum them and add on the number of days of the day variable in the input and/or access the calendar point for the relevant week starting sunday or monday and calculate or increment index forward to the correct day. Construct text string using fixed characters and insert the relevant variable such as the full name of the day of the week as required. There may be multiple traversals needed to obtain all the events of which the occurrences are to be displayed or counted.
First, find a sequence, if it exists:
step = {day,month,year}
period=0
for d = 1 to dates.count-1
interval(d,step)=datedifference(s,date(d),date(d+1))
next
' Find frequency with largest interval
for s = year downto day
found=true
for d = 1 to dates.count-2
if interval(d,s)=interval(d+1,s) then
found=false
exit for
end if
next
if found then
period=s
frequency=interval(1,s)
exit for
end if
next
if period>0
Select case period
case day
if frequency mod 7 = 0 then
say "every" dayname(date(1))
else
say "every" frequency "days"
end if
case month
say "every" frequency "months on day" daynumber(date(1))
case years
say "every" frequency "years on" daynumber(date(1)) monthname(date(1))
end select
end if
Finally, deal with "in November", "from 2007 to 2010" etc., should be obvious.
HTH
I like #arjen answer but I don't think there is any need for complex algorithm. This is so so simple. If there is a pattern, there is a pattern... therefore a simple algorithm would work. First we need to think of the types of patterns we are looking for: daily, weekly, monthly and yearly.
How to recognize?
Daily: there is a record every day
Weekly: there is a record every week
Monthly: there is a record every month
Yearly: there is a record every year
Difficult? No. Just count how many repetitions you have and then classify.
Here is my implementation
RecurrencePatternAnalyser.java
public class RecurrencePatternAnalyser {
// Local copy of calendars by add() method
private ArrayList<Calendar> mCalendars = new ArrayList<Calendar>();
// Used to count the uniqueness of each year/month/day
private HashMap<Integer, Integer> year_count = new HashMap<Integer,Integer>();
private HashMap<Integer, Integer> month_count = new HashMap<Integer,Integer>();
private HashMap<Integer, Integer> day_count = new HashMap<Integer,Integer>();
private HashMap<Integer, Integer> busday_count = new HashMap<Integer,Integer>();
// Used for counting payments before due date on weekends
private int day_goodpayer_ocurrences = 0;
private int day_goodPayer = 0;
// Add a new calendar to the analysis
public void add(Calendar date)
{
mCalendars.add(date);
addYear( date.get(Calendar.YEAR) );
addMonth( date.get(Calendar.MONTH) );
addDay( date.get(Calendar.DAY_OF_MONTH) );
addWeekendDays( date );
}
public void printCounts()
{
System.out.println("Year: " + getYearCount() +
" month: " + getMonthCount() + " day: " + getDayCount());
}
public RecurrencePattern getPattern()
{
int records = mCalendars.size();
if (records==1)
return null;
RecurrencePattern rp = null;
if (getYearCount()==records)
{
rp = new RecurrencePatternYearly();
if (records>=3)
rp.setConfidence(1);
else if (records==2)
rp.setConfidence(0.9f);
}
else if (getMonthCount()==records)
{
rp = new RecurrencePatternMonthly();
if (records>=12)
rp.setConfidence(1);
else
rp.setConfidence(1-(-0.0168f * records + 0.2f));
}
else
{
calcDaysRepetitionWithWeekends();
if (day_goodpayer_ocurrences==records)
{
rp = new RecurrencePatternMonthly();
rp.setPattern(RecurrencePattern.PatternType.MONTHLY_GOOD_PAYER);
if (records>=12)
rp.setConfidence(0.95f);
else
rp.setConfidence(1-(-0.0168f * records + 0.25f));
}
}
return rp;
}
// Increment one more year/month/day on each count variable
private void addYear(int key_year) { incrementHash(year_count, key_year); }
private void addMonth(int key_month) { incrementHash(month_count, key_month); }
private void addDay(int key_day) { incrementHash(day_count, key_day); }
// Retrieve number of unique entries for the records
private int getYearCount() { return year_count.size(); }
private int getMonthCount() { return month_count.size(); }
private int getDayCount() { return day_count.size(); }
// Generic function to increment the hash by 1
private void incrementHash(HashMap<Integer, Integer> var, Integer key)
{
Integer oldCount = var.get(key);
Integer newCount = 0;
if ( oldCount != null ) {
newCount = oldCount;
}
newCount++;
var.put(key, newCount);
}
// As Bank are closed during weekends, some dates might be anticipated
// to Fridays. These will be false positives for the recurrence pattern.
// This function adds Saturdays and Sundays to the count when a date is
// Friday.
private void addWeekendDays(Calendar c)
{
int key_day = c.get(Calendar.DAY_OF_MONTH);
incrementHash(busday_count, key_day);
if (c.get(Calendar.DAY_OF_WEEK) == Calendar.FRIDAY)
{
// Adds Saturday
c.add(Calendar.DATE, 1);
key_day = c.get(Calendar.DAY_OF_MONTH);
incrementHash(busday_count, key_day);
// Adds Sunday
c.add(Calendar.DATE, 1);
key_day = c.get(Calendar.DAY_OF_MONTH);
incrementHash(busday_count, key_day);
}
}
private void calcDaysRepetitionWithWeekends()
{
Iterator<Entry<Integer, Integer>> it =
busday_count.entrySet().iterator();
while (it.hasNext()) {
#SuppressWarnings("rawtypes")
Map.Entry pair = (Map.Entry)it.next();
if ((int)pair.getValue() > day_goodpayer_ocurrences)
{
day_goodpayer_ocurrences = (int) pair.getValue();
day_goodPayer = (int) pair.getKey();
}
//it.remove(); // avoids a ConcurrentModificationException
}
}
}
RecurrencePattern.java
public abstract class RecurrencePattern {
public enum PatternType {
YEARLY, MONTHLY, WEEKLY, DAILY, MONTHLY_GOOD_PAYER
}
public enum OrdinalType {
FIRST, SECOND, THIRD, FOURTH, FIFTH
}
protected PatternType pattern;
private float confidence;
private int frequency;
public PatternType getPattern() {
return pattern;
}
public void setPattern(PatternType pattern) {
this.pattern = pattern;
}
public float getConfidence() {
return confidence;
}
public void setConfidence(float confidence) {
this.confidence = confidence;
}
public int getFrequency() {
return frequency;
}
public void setFrequency(int frequency) {
this.frequency = frequency;
}
}
RecurrencePatternMonthly.java
public class RecurrencePatternMonthly extends RecurrencePattern {
private boolean isDayFixed;
private boolean isDayOrdinal;
private OrdinalType ordinaltype;
public RecurrencePatternMonthly()
{
this.pattern = PatternType.MONTHLY;
}
}
RecurrencePatternYearly.java
public class RecurrencePatternYearly extends RecurrencePattern {
private boolean isDayFixed;
private boolean isMonthFixed;
private boolean isDayOrdinal;
private OrdinalType ordinaltype;
public RecurrencePatternYearly()
{
this.pattern = PatternType.YEARLY;
}
}
Main.java
public class Algofin {
static Connection c = null;
public static void main(String[] args) {
//openConnection();
//readSqlFile();
RecurrencePatternAnalyser r = new RecurrencePatternAnalyser();
//System.out.println(new GregorianCalendar(2015,1,30).get(Calendar.MONTH));
r.add(new GregorianCalendar(2015,0,1));
r.add(new GregorianCalendar(2015,0,30));
r.add(new GregorianCalendar(2015,1,27));
r.add(new GregorianCalendar(2015,3,1));
r.add(new GregorianCalendar(2015,4,1));
r.printCounts();
RecurrencePattern rp;
rp=r.getPattern();
System.out.println("Pattern: " + rp.getPattern() + " confidence: " + rp.getConfidence());
}
}
I think you'll have to build it, and I think it will be a devil in the details kind of project. Start by getting much more thorough requirements. Which date patterns do you want to recognize? Come up with a list of examples that you want your algorithm to successfully identify. Write your algorithm to meet your examples. Put your examples in a test suite so when you get different requirements later you can make sure you didn't break the old ones.
I predict you will write 200 if-then-else statements.
OK, I do have one idea. Get familiar with the concepts of sets, unions, coverage, intersection and so on. Have a list of short patterns that you search for, say, "Every day in October", "Every day in November", and "Every day in December." If these short patterns are contained within the set of dates, then define a union function that can combine shorter patterns in intelligent ways. For example, let's say you matched the three patterns I mention above. If you Union them together you get, "Every day in October through December." You could aim to return the most succinct set of unions that cover your set of dates or something like that.
Have a look at your favourite calendar program. See what patterns of event recurrence it can generate. Reverse engineer them.

Resources