Datastructure for a log search service? - data-structures

A few months back, i was asked to design a service which takes a start and end time interval and lists down the number of exceptions/errors grouped by the exception type/code. Basically the intention was to create or use an existing data structure for an efficient search. Here is what I coded.
public class ErrorDetail{
private int code;
private String message;
}
public class ExceptionSearchService
{
private Map<Long, ErrorDetail> s = new TreeMap<Long, ErrorDetail>();
public ArrayList<ErrorDetail> getErrors(long start, long end){
ErrorDetail e1 = find(start, s);
ErrorDetail e2 = find(end, s);
//do an in order traversal between e1 and e2 and add it to an array list and return
}
public void addError(long time, ErrorDetail e){
s.put(time,e);
}
}
I realized that i should not have mentioned a TreeMap, instead should have had my own class like TreeNode but the idea was to have a tree structure and a distributed one because we are talking about thousands of services serving millions of request per minute and generating error.
Could i have used a better data structure in this case ?

Related

Feature envy, encapsulation, active record, separation of concerns? When its bad?

you all say, object oriented programming is about encapsulation, data hiding. Let's given this example:
class Rectangle
{
private int a,b;
public function __construct(int a, int b)
{
this.a = a;
this.b = b;
}
int public function getA()
{
return a;
}
int public function getB()
{
return b;
}
}
var r = new Rectangle(3, 4);
var area = r.getA() * r.getB();
this is a bad code then, so let's refaktor:
class Rectangle
{
private int a,b;
public function __construct(int a, int b)
{
this.a = a;
this.b = b;
}
int public function getArea()
{
return a*b;
}
}
r = new Rectangle(3, 4);
area = r.getArea();
way better, data hiding is done and getArea is brought where it belongs to.
Ok then, here comes the Active Records:
class Record
{
private int ID;
private string username;
public function __constructor(int ID, string username)
{
this.ID = ID;
this.username = username;
}
int public function getID()
{
return ID;
}
string public function getUsername()
{
return username;
}
}
r = new Record(1, 'test');
dbEngine.save(r);
this is again bad, since all data is public. (altough Doctrine works this way)
But if I do that as Propel did:
class Record
{
private int ID;
private string username;
public function __constructor(int ID, string username)
{
this.ID = ID;
this.username = username;
}
public function save()
{
dbEngine.save([ID, username]);
}
}
r = new Record(1, 'test');
r.save();
this is also said bad, because Active Records are antipattern. Then when it's good or bad? When does an "act" (getArea, save) should be brought inside an object - and when does it act outsidely?
You can inject the dbEngine dependency in for your specific case, but this doesn't address your concern.
In general, what makes your code good is how easy it is to understand, and how close changes in intention are tied to changes in implementation.
The problem with revealing private internals are that you're exposing your inner values that programs which interface with your program may rely on (and make difficult to change later on). A record is basically a struct/dataclass - it represents a collection of values that goes together with some well-defined meaning. Without knowing the rest of the code I can't say if this specific class is like that, but if that's the case it would be okay to just make it a struct (all members public, no methods).
There aren't any catch-all rules that makes code 'good'. It's a continuous process of making mistakes or being inefficient, and analysing what code led or made more likely that problem. Code smells are just the result of lots of trial and error by others, and although very robust in most cases may sometimes be outdated and should be applied in the specific situation when they improve your code.
None of your examples are bad. They are just design choices. Dropping the accessors to a and b in the second example seems a step backwards to me. As to putting implementation dependent save code in the class definition, that would be bad if there were multiple types that all needed to define the save. There you would be better to define a parent class with the save function and then inheriting from that class. However, if it’s just you writing code and there is just that one class it doesn’t matter.
Good that you are thinking about what makes good code. As a general rule, good code is code that works and which you can return to in 6 months and modify easily in the future. If you have a group of developers then of course provide accessors.
Another aspect of good code is having unit tests. If you change something and the unit tests pass you’ve done your job. If someone is using internals they should write a unit test that will signal a change that would break their code.

Gson IllegalStateException: Expected an int but was BEGIN_ARRAY at line 1 column Y

Since I've added an integer to my Schedule class, Gson is throwing an error on some devices: java.lang.IllegalStateException: Expected an int but was BEGIN_ARRAY at line 1 column Y (e.g. column 112 or 120 etc). I looked at this and this answer, which seems to suggest Gson is expecting an int but is getting a BEGIN_ARRAY char, but I have no idea why this would happen after the refactor of adding an extra int to the class.
Before this, my code to parse the list of Schedule objects from a stored Json string was working perfectly fine. I added the Since annotation because of the exception being thrown. Here's the Schedule class:
public class Schedule {
/**
* Added this variable
*/
#Since(1.1) private int addedVar;
/**
* All other variables have the #Since(1.0) annotation
*/
#Since(1.0) all other vars;
}
The function to parse the schedules:
public static ArrayList<Schedule> schedulesFromJson(String schedulesJson) {
Type listType = new TypeToken<ArrayList<Schedule>>(){}.getType();
Gson gson = new Gson();
try {
return gson.fromJson(schedulesJson, listType);
} catch (Exception exception) {
// Try to use the previous version of the schedule, because of IllegalStateException
gson = new GsonBuilder().setVersion(1.0).create();
return gson.fromJson(schedulesJson, listType);
}
}
The strange thing is: on some devices (like my own test devices), this crash never happened. Because of the crash, I added the Since annotation with the try and catch clause, since I expected it might have to do with the extra integer being added and could prevent that by simply reading in the old Schedule version, but this is still throwing the same exception in the catch clause.
Any help with why this is happening?
Figured it out: Because Proguard wasn't set up to not obfuscate the Schedule object (thanks #Marcono1234 for tipping me in the right direction), the Schedule object was stored in storage as an obfuscated object ({"a":true,"b":"Name","c":[true,true,true,true,true,false,false], etc}) instead of using the variable names.
The Exception was thrown because, based on the Schedule class structure before adding the addedVar, there was an array in the schedule. Easier with example.
The old schedule class:
public class Schedule {
private boolean isActive;
private String scheduleName;
private boolean[] days;
private final long timeCreated;
private ArrayList<String> list;
}
The new schedule class:
public class Schedule {
private boolean isActive;
private String scheduleName;
private boolean[] days;
private final long timeCreated;
private int addedVar; // <-- Here it goes wrong
private ArrayList<String> list;
}
Because of adding the int variable before the ArrayList<String> list, when Gson tried to deserialize the stored JSON String, it expected to see an int (the addedVar, but instead saw BEGIN_ARRAY, from the list.
I fixed it by placing the `addedVar`` after the list in my Schedule class, so the Since(1.0) annotation will properly work.

How can I test a session method?

I'm trying to #Test a Service class but there are several get() methods that I don't know how to test. I would need to know how to collect the data that is necessary or at least how to test the rest of the methods of the TokenHelper class.
This is the Session class:
public class SessionData {
public static final String KEY = "session_data";
private Integer id;
private String email;
private String fullName;
private List<Role> role;
private Boolean tempSession;
private int permissionsMask = 0;
private String avatar;
public boolean hasAnyRole(Role... roles) {
for (Role r : roles) {
if (this.role.contains(r)) {
return true;
}
}
return false;
}
}
This is the TokenHelper class:
public class TokenHelper {
public String generate(SessionData tokenData, long expirationInHours) {
return Jwts.builder()
.claim(SessionData.KEY, tokenData)
.setIssuedAt(Date.from(Instant.now()))
.setExpiration(Date.from(Instant.now().plus(expirationInHours, ChronoUnit.HOURS)))
.signWith(SignatureAlgorithm.HS256, TextCodec.BASE64.encode(secret))
.compact();
}
public UserGoogle getTokenDataFromGoogleToken(String token) throws InvalidTokenException {
try {
int i = token.lastIndexOf('.');
String withoutSignature = token.substring(0, i + 1);
Claims claims = Jwts.parser().parseClaimsJwt(withoutSignature).getBody();
return UserGoogle.builder()
.email(claims.get(UserGoogle.KEY_EMAIL).toString())
.firstName(claims.get(UserGoogle.KEY_FIST_NAME).toString())
.lastName(claims.get(UserGoogle.KEY_LAST_NAME).toString()).build();
} catch (ExpiredJwtException | MalformedJwtException | SignatureException | IllegalArgumentException ex) {
log.error(ERROR_TOKEN, ex.toString());
throw new InvalidTokenException();
}
}
}
This is my #Test:
#Test
void googleTokenHelperTest() throws InvalidTokenException {
TokenHelper obj1 = BeanBuilder.builder(TokenHelper.class).createRandomBean();
String mailGoogle = "google#prueba.com";
String firstGoogle = "Nombre";
String lastGoogle = "Apellido";
Map<String, Object> pruebaGoogle = new HashMap<String, Object>();
List<String> info = new ArrayList<String>();
info.add(firstGoogle);
info.add(lastGoogle);
pruebaGoogle.put(mailGoogle, info);
UserGoogle expectedUser = UserGoogle.builder().email(mailGoogle).firstName(firstGoogle).lastName(lastGoogle).build();
String myTestToken = pruebaGoogle.toString();
UserGoogle actualUser = obj1.getTokenDataFromGoogleToken(myTestToken);
assertEquals(actualUser, expectedUser);
}
I have created some variables to form a user, but I need to build them with a map to generate the token with the help of the generate () method. I need to know how to join those three variables and pass them to the generate () method, and then pass the result variable to the google method to generate the new user.
Edit: After clarification by OP the topic of the question changed.
Your problem arises from a flawed Object-Orientation-Design. For example, your SessionData implicitly holds a User by having String-fields relevant to a User among fields relevant to a Session. This overlapping makes it hard to test your code, because in order to test your Token-Generation for some User data, you need a Session object, which introduces additional data and dependencies.
That is one major reason, why it's difficult for you, to get a token from your three input values.
You want to test getTokenDataFromGoogleToken(String token). First thing you need to know is, what a valid Token-String will look like.
Next, you will need to mock your Claims claims object in one of two ways:
Mockito.mock it using Mockito to return the necessary Strings when claims.get() is called.
Mockito.mock your Jwts.parser().parseClaimsJwt(withoutSignature).getBody() to return a Claims object that serves your testing purpose.
Since the signature of your token will be irrelevant to your tested method, just focus on the substring before the .-Separator, i.e. the part after . in your token string can be any string you like.
If you want to test generate(SessionData, long) you need to supply a SessionData Object and a long value. After that you assertEquals the String as necessary. However, currently your code does not imply that your get is in any way related to your generate. This is, because you just handle Strings. A better design would be to have e.g. a User, Session and Token-classes, which would also make it easier to test your application and units.
A Test for your getToken method looks like the following, you just have to replace ... with your test data.
#Test
void givenGoogleToken_whenTokenHelperGeneratesUserFromToken_UserOk() {
TokenHelper helper = new TokenHelper();
String myTestToken = ...; //
UserGoogle expectedUser = ... // generate the UserGoogle Object you expect to obtain from your TokenHelper class
UserGoogle actualUser = helper.getTokenDataFromGoogleToken(myTestToken);
assertEquals(actualUser, expectedUser);
}
Test generally follow a given-when-then structure. Given some precondition, when some action is performed, then some result is returned/behaviour observed. When implemented very formally, this is called BDD (Behaviour Driven Development), but even when not practicing BDD, tests still generally follow that pattern.
In this case, I would suggest the tests be something like:
Given some data exists in the service threaddata
when I call get
then I get back the expected value
In the scenario above, the given part probably consists of setting some data on the service, the when is invoking get and the then is asserting that it's the expected value.
And I'd encourage you to consider the various scenarios. E.g what happens if the data isn't there? what happens if it's not the class the consumer asks for? Is the map case-sensitive? etc...
Code sample for the initial instance (I'm not sure what BeanBuilder is here, so I've omitted it):
#Test
public void testCurrentThreadServiceReturnsExpectedValue() {
final String key = "TEST KEY";
final String value = "TEST VALUE";
//Initialize System Under Test
CurrentThreadService sut = new CurrentThreadService();
//Given - precondition
sut.set(key, value);
//When - retrieve value
String observedValue = sut.get(key, String.class);
//Then - value is as expected
assertEquals(value, observedValue);
}
EDIT TO ADD It's always great to see someone get into unit testing, so if you have any follow ups, please ask I'm happy to help. The confidence one derives from well tested code is a great thing for software devs.

LRU Cache Key, Value, Node.Value Real World Interpretation

I understand how in principle an LRU cache works. For example, see here: https://java2blog.com/lru-cache-implementation-java/
However, I am having great difficulty understanding how this is interpreted in a real world setting. For example, if I want to store objects (which have no natural numbering/order), I understand that the value (in the hashmap) is just a pointer to a node in the linked list, but what does the key represent?
Furthermore, what does the node.value represent? I think this the actual object which is being cached. However, how does this correspond to the key in the hashmap?
A typical hashmap has a key and a value, both of arbitrary type. The key is the thing you want to index the structure by, and the value is the thing you want to store and retrieve. Consider a normal hashmap in Java:
Map<UUID, Person> peopleById = new HashMap<>();
You can pass in a UUID to a .get method and get the person associated with that UUID, if it exists.
The LRU caches used in the real world are like that as well:
Map<UUID, Person> cachedPeopleById = new LRUCache<>(10);
The UUID is the key, and the Person is the value.
The reference implementation you linked to doesn't use generics, it only supports int to int, which is the equivalent of Map<Integer, Integer>. The Node class in the reference implementation isn't something that ought to be exposed in public methods. So in that reference implementation, Node should be hidden, and delete(Node) and setHead(Node) should be private, because otherwise they expose implementation details of the cache.
A better implementation would be something more like this (doing this off the top of my head, might have compilation errors, for illustrative purposes only):
public class LRUCache <KeyType, ValueType> implements Map<KeyType, ValueType> {
private static class Node <KeyType, ValueType> {
KeyType key;
ValueType value;
Node prev;
Node next;
public Node(KeyType key, ValueType value){
this.key = key;
this.value = value;
}
}
int capacity;
HashMap<KeyType, Node> map = new HashMap<>();
Node head=null;
Node end=null;
public LRUCache(int capacity) {
this.capacity = capacity;
}
public ValueType get(KeyType key) {
...
}
public set(KeyType key, ValueType value) {
...
}
private void delete(Node<KeyType, ValueType> node) {
...
}
private void setHead(Node<KeyType, ValueType> node) {
...
}

OCPJP 8 II Consumer vs Supplier IZ0-809

I am trying to get IZO-809 certification I was reading the OCA/OCP SE8 test book and a code really caught my attention.
The code gets me to this question.
I know consumer get a parameter and not return nothing and Supplier has not parameters and returns a value.
But this code is almost the same after the ->.
public class Pregunta24{
private final Object obj;
public Pregunta24(final Object obj){
this.obj = obj;
}
}
//Returns a Supplier
private final Supplier<Pregunta24>supplier = ()->new Pregunta24("HI");
//Returns a Consumer.
private final Consumer<Pregunta24>consumer = a->new Pregunta24(a);
Both codes work.
But if this code not work i know that consumer doesn't return nothing.
private final Consumer<String>consumerString = String::length
I know this not work because consumer doesn't return a value my question is in the supplier code and the consumer code the code is right after the -> mark but this time is considered return in fact a instance of the class.
My question is why sometimes Java complaints that is a return value and something not?
I mean this code.
private final Supplier<Pregunta24>supplier = ()->new Pregunta24("HI");
// I would think is returning a instance of the Pregunta24 class.
private final Consumer<Pregunts24>consumer = a->new Pregunta24(a);
Is returning the same after the -> but why in the consumer don't say the error.
incompatible types: bad return type in lambda expression
But if do this I do
final Consumer<String>consumerString = a->1;
I think the code after the -> is context inferred.
According to javadoc Consumer:
Represents an operation that accepts a single input argument and
returns no result.
Consumer<Pregunts24>consumer = a->new Pregunta24(a);
doesn't actually return anything. This basically is implementation of Consumer#accept method, which accepts an object of type T and has void as return type.
public void accept(Pregunta24 a) {
new Pregunta24(a);
}
You are not returning anything. Same thing with
Consumer<String>consumerString = String::length
public void accept(String a) {
a.length();
}
However
Consumer<String>consumerString = a->1;
is an invalid expression which is translated to something like this:
public void accept(String a) {
1;
}

Resources