Why is my Kafka Transformer's StateStore not accessible? - apache-kafka-streams

I am trying to implement a Transformer class
public class StreamSorterByTimeStampWithDelayTransformer < V >
implements Transformer< Long, V, KeyValue< Long, V > >
The constructor for the class creates a StateStore for each instance, thus:
this.state_store_name = "state_store_" + this.hashCode();
KeyValueBytesStoreSupplier key_value_store_supplier = Stores.persistentKeyValueStore( state_store_name );
StoreBuilder< KeyValueStore< String, V > > key_value_store_builder =
Stores.keyValueStoreBuilder( key_value_store_supplier, Serdes.String(), v_instance.getSerde() );
stream_builder.addStateStore( key_value_store_builder );
The Transformer init method references the StateStore thus:
public void init(ProcessorContext context) {
this.context = context;
this.key_value_store = (KeyValueStore< String, V >) context.getStateStore( state_store_name );
// schedule a punctuate() method every "poll_ms" (wall clock time)
this.context.schedule( this.poll_ms, PunctuationType.WALL_CLOCK_TIME,
(timestamp) -> pushOutOldEnoughEntries() );
}
I think I must be leaving out a step because when the getStateStore call is made, it
results in an exception saying:
Processor KSTREAM-TRANSFORM-0000000003 has no access to StateStore
What am I omitting or doing wrong?

You need to attach the store to the transformer:
stream.transform(..., this.state_store_name);

Related

Quarkus prometheus percentiles still zeroed after long metrics update time

I have a service bean MonitoringRegistryServiceImpl implementation in the Quarkus which I used to register timers and update them (code bellow). They were registered only once (into map timerMap) and then reused when recording / updating their values.
The problem I had that if I called http://localhost:8080/q/metrics to see the current state of percentile metrics I saw only zero values.
This happens only after long time period (e.g.3 hours without update of metrics). So far so good, that is something I would expect as there is distributionStatisticExpiry(Duration.ofMinutes(5)) for the percentiles. Let's call it "natural decay of values" :)
However, if I would updated the metrics and then called the http://localhost:8080/q/metrics again the zero values on percentiles were still there yet related counters and max values were updated correctly.
Maybe my understanding of percentiles is not complete, please advice.
Note: I use quarkus-micrometer-registry-prometheus artifact.
#JBossLog
#ApplicationScoped
public class MonitoringRegistryServiceImpl implements MonitoringRegistryService {
private static final double[] PERCENTILES = { 0.5, 0.75, 0.95, 0.99 };
private Map<String, Timer> timerMap = new ConcurrentHashMap<>();
#Inject
MeterRegistry registry;
#Override
public <T> void updateTimer(String timerName, String[] tags,
Duration duration) {
String key = timerName + "_" + String.join(".", tags);
Timer timer = timerMap.get(key);
if (timer != null) {
log.debug("Key found:" + key);
timer.record(duration);
} else {
log.debug("Key not found:" + key);
Timer newTimer = Timer.builder(timerName).tags(newTags).publishPercentiles(PERCENTILES)
.distributionStatisticExpiry(Duration.ofMinutes(5))
.register(registry);
timerMap.put(key, newTimer);
newTimer.record(duration);
}
}
}
The solution was quite simple, rather than using the Timer.builder()..register() just use registry.timer(). There was also no need for the timerMap above. After this change, the timers metrics were not zeroed.
Timer timer = registry.timer(timerName, tags);
if (duration != null) {
timer.record(duration);
} else {
log.error("Time duration was equal to null for a metric with a timerName:" + timerName);
}

How to check record in c# 9 in NET 5 is immutable at runtime

Record is a new feature in c#9, Net 5
It's said
If you want the whole object to be immutable and behave like a value, then you should consider declaring it as a record
Creating a record in c#9 , NET 5:
public record Rectangle
{
public int Width { get; init; }
public int Height { get; init; }
}
Then instantiating it:
var rectangle = new Rectangle (20,30);
Trying to change the value:
rectange.Width=50; //compiler error
Compiler raise the error:
error CS8852: Init-only property or indexer 'Rectangle.Width' can only be assigned in an object initializer, or on 'this' or 'base' in an instance constructor or an 'init' accessor.
That is right and insure that the record is immutable.
Using a method like to test IsImmutable type give false, because in record there is no generated readonly properties.
How to check the record in c# 9, Net 5 is immutable at runtime or even it has init property?
A record is indeed mutable at runtime. This is intentional, is it means most serializer frameworks work without updating.
It is however possible to check if a property is initonly by checking:
public static bool IsInitOnly(PropertyInfo propertyInfo)
{
return propertyInfo?.SetMethod.ReturnParameter
.GetRequiredCustomModifiers()
.Any(x => x.FullName == _isExternalInitName)
?? false;
}
private static string _isExternalInitName =
typeof(System.Runtime.CompilerServices.IsExternalInit).FullName;
I don't think that it's possible to check for immutability at runtime.
Here's some of the generated code for your record. You can see that both properties have a public setter.
public class Rectangle : IEquatable<Rectangle>
{
[CompilerGenerated]
private readonly int <Width>k__BackingField;
[CompilerGenerated]
private readonly int <Height>k__BackingField;
protected virtual Type EqualityContract
{
[CompilerGenerated]
get
{
return typeof(Rectangle);
}
}
public int Width
{
[CompilerGenerated]
get
{
return <Width>k__BackingField;
}
[CompilerGenerated]
set
{
<Width>k__BackingField = value;
}
}
public int Height
{
[CompilerGenerated]
get
{
return <Height>k__BackingField;
}
[CompilerGenerated]
set
{
<Height>k__BackingField = value;
}
}
The following code will compile and run without errors.
var rect = new Rectangle { Height = 1, Width = 2 };
typeof(Rectangle).GetProperty("Height").SetValue(rect, 5);
Console.Write(rect.Height);
//Prints 5
At runtime the init accessor is just a regular setter. It's only at compile time that a check is made to only allow init accessor to be called during object initialization.
So I don't see any way to check at runtime that Rectangle is immutable.

Returning reference to slice_array

I'm currently trying to implement a class around a valarray object and i get a exception when trying to run:
class TestClass
{
public:
valarray<int> va;
TestClass() { va.resize(5, 1); }
slice_array<int>& get_slice()
{
return va[slice(0, 3, 2)];
}
};
//In main():
TestClass obj;
slice_array<int>& ref = obj.va[slice(0,3,2)];
ref = -1; //OK
obj.get_slice() = -1; //Throws exception
Aren't the two assignments the same thing?
I solved it myself:
It seems that a valarray[slice(int, int, int)] is not an lvalue or is temporary, so it's not allowed to initialize a reference with such an object (oddly enough, VS2012 allows to do so in some cases).

Extension method with generic Func parameter of other type

Is there a way to implement an extension method to a generic type that takes in arguments a Func of another type?
For exemple, a usage something similar to this:
myFirstObject.Extension<myOtherObject>( other => other.Prop );
Or with a more complicated Func:
myFirstObject.Extension<myOtherObject>( other => other.Prop > 2 && other.Prop < 15 );
I found some related question like this one, but in my case, I need generic types inside the extension method too.
Here's what I came up with:
public static bool Extension<TSource, TIn, TKey>(this TSource p_Value, Expression<Func<TIn, TKey>> p_OutExpression)
{ return true; }
However, when I try to use it, it does not take into consideration the second type.
Am I missing something?
Look at this:
s => s.Length;
How's the compiler suppose to know whether or not s is a string or s is an array or some other type that has a Length property? It can't, unless you give it some information:
(string s) => s.Length;
Oh, there we go. So now, try this:
myFirstObject.Extension((myOtherObject o) => o.Prop > 2 && o.Prop < 15);
That will work, because you've told the compiler what it should use for TIn, and it can figure out what to use for TKey based on the expression.
I found that another solution would be to create another method that takes in argument a type.
For instance:
Void Extension(Type p_Type, [THE TYPE] p_Params)
{
MethodInfo realExtensionMethod = typeof([CLASS CONTAINING THE METHOD]).GetMethod("RealExtension");
realExtensionMethod = realExtensionMethod.MakeGenericMethod(p_Type);
realExtensionMethod.Invoke(null, new object[] {p_Type, p_Params });
}
Void RealExtension<TYPE>(params)
{
}
Then at usage time:
Type objectType = typeof(myOtherObject);
myFirstObject.Extension(objectType, other => other.Prop );
When you call a generic method in C# you can explicitly declare all of the generic type parameters or you can have them all inferred, but you cannot have some explicitly declared and some inferred.
So, if I had this method:
public void Foo<X, Y>(X x, Y y)
{
/* Do somethhing */
}
Then here's what works and what doesn't:
int a = 42;
string b = "Hello, World!";
// Legal
Foo(a, b);
Foo<int, string>(a, b);
//Illegal
Foo<int>(a, b);
The best you can do is move the first generic parameter up to the class level, but not it won't work as an extension method. Nevertheless you may like this approach.
public static class Class<TSource>
{
public static bool Method<TIn, TKey>(
TSource p_Value,
Expression<Func<TIn, TKey>> p_OutExpression)
{
return true;
}
}
Now you can call it like this:
Expression<Func<long, decimal>> f =
l => (decimal)l;
var result = Class<int>.Method(a, f);
But as I say, it won't work as an extension method now.

JavaFX Events Using Local Variables

I have a method that creates an ImageView, called createImageView(), which uses EventHanlders to call vaious events. I would like to be able to use local variables in this method. What I usually do is assign final variables to use within the handle(Event) method.
In the method described below, when I go to use final I get the following errors:
The final local variable initY cannot be assigned, since it is defined in an enclosing type
The final local variable initY cannot be assigned, since it is defined in an enclosing type
The final local variable dragAnchor cannot be assigned, since it is defined in an enclosing type
Of course, removing final gives me this message:
Cannot refer to a non-final variable initX inside an inner class defined in a different method
So what do I have to do to make this method work?
private ImageView createImageView() {
double initX;
double initY;
Point2D dragAnchor;
imgView.setOnMousePressed(new EventHandler < MouseEvent > () {
public void handle(MouseEvent me) {
//when mouse is pressed, store initial position
initX = imgView.getTranslateX();
initY = imgView.getTranslateY();
dragAnchor = new Point2D(me.getSceneX(), me.getSceneY());
me.consume();
}
});
imgView.setOnMouseDragged(new EventHandler < MouseEvent > () {
public void handle(MouseEvent me) {
double dragX = me.getSceneX() - dragAnchor.getX();
double dragY = me.getSceneY() - dragAnchor.getY();
double newXPosition = initX + dragX;
double newYPosition = initY + dragY;
imgView.setTranslateX(newXPosition);
imgView.setTranslateY(newYPosition);
me.consume();
}
});
imgView.setOnMouseEntered(new EventHandler < MouseEvent > () {
public void handle(MouseEvent event) {
imgView.setEffect(new Glow(0.5));
event.consume();
}
});
imgView.setOnMouseExited(new EventHandler < MouseEvent > () {
public void handle(MouseEvent event) {
imgView.setEffect(new Glow(0.0));
event.consume();
}
});
return imgView;
}
Use JavaFX properties instead of primitive types.
For example:
final DoubleProperty initX = new SimpleDoubleProperty();
final ObjectProperty<Point2D> dragAnchor = new SimpleObjectProperty<>();
...
// in event handler
initX.set(
imageView.getTranslateX()
);
dragAnchor.set(
new Point2D(
me.getSceneX(),
me.getSceneY()
)
);
I removed the 3 local variables and made them class variables and it worked. If anyone doesn't mind offering a quick explanation as of why I would appreciate it.

Resources