Check if redis key is hash or string using restTemplate - spring-boot

I have Redis DB consisting of hashes and strings.
I got all keys from DB using this code:
Set<byte[]> keys = redisTemplate.getConnectionFactory().getConnection().keys("*".getBytes());
Iterator<byte[]> it = keys.iterator();
while(it.hasNext()){
byte[] data = (byte[])it.next();
String key = (new String(data, 0, data.length));
System.out.println(key);
}
from here: How to get all Keys from Redis using redis template
Since the key can be hash or string, how to determine when I can use opsForHash and opsForValue i.e. how to check if it is hash or it is a string in spring boot using restTemplate.

The Redis command to get the type of a key is TYPE: https://redis.io/commands/type
You can use RedisTemplate's public DataType type(K key) method to execute this: https://docs.spring.io/spring-data/redis/docs/current/api/org/springframework/data/redis/core/RedisTemplate.html#type-K-
Here's an example:
Set<byte[]> keys = redisTemplate.getConnectionFactory().getConnection().keys("*".getBytes());
Iterator<byte[]> it = keys.iterator();
while(it.hasNext()){
byte[] data = (byte[])it.next();
String key = (new String(data, 0, data.length));
DataType type = redisTemplate.type(key);
if (type == DataType.HASH) {
// ...
} else if (type == DataType.STRING) {
// ...
}
}
Edit: one more piece of advice is that you may want to use SCAN instead of KEYS * (mentioned in one answer to the SO question you linked). Scan is generally better in production since it doesn't try to get and return all keys at once.

Related

Keep an object as a value in Redis using its HashTable

I am new to Redis and I'm trying to write a simple project that collects information from SQL database and caches into Redis. As I'm more comfortable with C#, I've chosen StackExchange.Redis to do that.
Let's say I have a table in my db with a schema like this Persons(ID, Name, Address, Age, BirthDate).
I have a Person class in my project with corresponding fields.
I have also a function GetPersonByID(ID), that requests the Redis, if a key with the ID doesn't exist it executes another function called GetPersonByID_SQL(ID), when an sql query is being executed, after getting information from db it creates an object Person, adds that object to Redis(using hashTable) and returns the object. If the key existed in Redis the function just gets information from Redis, creates an object Person, maps the corresponding values values to fields and returns that object.
Here is the code of how I do this.
public static Person GetPersonByID(string ID)
{
redis = ConnectionMultiplexer.Connect("127.0.0.1");
IDatabase db = redis.GetDatabase();
Person p;
if (!db.KeyExists(key))
{
p = Person.GetPersonByID_SQL(ID);
db.HashSet(key, "Name", p.Name);
db.HashSet(key, "Address", p.Address);
db.HashSet(key, "Age", p.Age);
db.HashSet(key, "BirthDate", p.BirthDate);
}
else
{
HashEntry[] values = db.HashGetAll(key);
p = new Person();
for (int i = 0; i < values.Length; i++)
{
HashEntry hashEntry = values[i];
switch (hashEntry.Name)
{
case "Name": p.Name = hashEntry.Value; break;
case "Address": p.Address = hashEntry.Value; break;
case "Age": p.Age = hashEntry.Value; break;
case "BirthDate": p.BirthDate = hashEntry.Value; break;
}
}
}
return p;
}
My question is, Is there any way I can Bind automatically the value of Redis (that is in my way a HashTable) to my existing object?
My question is, Is there any way I can Bind automatically the value of Redis (that is in my way a HashTable) to my existing object?
No, this is not a feature of StackExchange.Redis - values can only be stored as the most basic types (strings and numbers). You're expected to do the conversions to more complex values yourself.
So you can either store your person object as multiple hash fields (as you've done in your code), or you can store your person object as a single serialized string against a key (you can use the STRING data structure for this instead). Then you can perform the necessary deserializations when you want to retrieve a value.
If you always want to retrieve every value from your Person data structure, I would recommend going for the second option. You'll only then need to use a single command command for each get/set:
// SET
_redis.StringSet(key, serializedPerson);
// GET
string serializedPerson = _redis.StringGet(key);

grpc and protobuf with variable keys

I have a hash of key values pairs.
I do not have the keys, nor the values.
I can assume most will be there, but sometimes keys are removed and others are added. Is there any way for me to have a message with variable keys?
{
"knownkey1": "value",
"knownkey2": {
"unknown-key1": "value",
"unknown-key2": "value"
}
}
or is the best ways to just serialize it using json stringify in the message? i would think that would defeat the whole purpose of using grpc.
message Rate {
string ticker = 1;
string value = 2;
}
message GetAllResponse {
string lastUpdated = 1;
repeated Rate payload = 2;
}
Looks like you can just use the maps type as outlined here:
https://developers.google.com/protocol-buffers/docs/proto3#maps

Castle Core Invocation create a cache key from intercepted method

I'm using a interface interceptor to cache all methods that starts with "Get" but i can't figure out how to generate a unique cache key for every unknown parameter it can be anything and using GetHashCode is not an option as i can't be 100% sure that they have overridden the GetHashCode.
some thoughts was someting in the line of How can I create a unique hashcode for a JObject?
where JSON is used for a JObject i was thinking on JSON serialize every parameter then get the hash code like explained in the link above:
var obj = JToken.Parse(jsonString);
var comparer = new JTokenEqualityComparer();
var hashCode = comparer.GetHashCode(obj);
However i think this will be a performence hit so how can this be solved ?
The code i have so far is this but it wont handle the complex type situation where .ToString won't generate the raw value type like int, string etc.
private string CreateCacheKey(IInvocation invocation)
{
string className = invocation.TargetType.FullName;
string methodName = invocation.Method.Name;
var builder = new StringBuilder(100);
builder.Append(className);
builder.Append(".");
builder.Append(methodName);
for (int i = 0; i < invocation.Arguments.Length; i++)
{
var argument = invocation.Arguments[i];
var argumentValue = invocation.GetArgumentValue(i);
builder.Append("_");
builder.Append(argument);
if (argument != argumentValue)
{
builder.Append(argumentValue);
}
}
return string.Format("{0}-{1}", this.provider.CacheKey, builder);
}
I ended up using GetHashCode as it is the only viable solution if thay dont override the method it will not cache.

Spark RDD to update

I am loading a file from HDFS into a JavaRDD and wanted to update that RDD. For that I am converting it to IndexedRDD (https://github.com/amplab/spark-indexedrdd) and I am not able to as I am getting Classcast Exception.
Basically I will make key value pair and update the key. IndexedRDD supports update. Is there any way to convert ?
JavaPairRDD<String, String> mappedRDD = lines.flatMapToPair( new PairFlatMapFunction<String, String, String>()
{
#Override
public Iterable<Tuple2<String, String>> call(String arg0) throws Exception {
String[] arr = arg0.split(" ",2);
System.out.println( "lenght" + arr.length);
List<Tuple2<String, String>> results = new ArrayList<Tuple2<String, String>>();
results.addAll(results);
return results;
}
});
IndexedRDD<String,String> test = (IndexedRDD<String,String>) mappedRDD.collectAsMap();
The collectAsMap() returns a java.util.Map containing all the entries from your JavaPairRDD, but nothing related to Spark. I mean, that function is to collect the values in one node and work with plain Java. Therefore, you cannot cast it to IndexedRDD or any other RDD type as its just a normal Map.
I haven't used IndexedRDD, but from the examples you can see that you need to create it by passing to its constructor a PairRDD:
// Create an RDD of key-value pairs with Long keys.
val rdd = sc.parallelize((1 to 1000000).map(x => (x.toLong, 0)))
// Construct an IndexedRDD from the pairs, hash-partitioning and indexing
// the entries.
val indexed = IndexedRDD(rdd).cache()
So in your code it should be:
IndexedRDD<String,String> test = new IndexedRDD<String,String>(mappedRDD.rdd());

ASP.NET MVC - Integrating zetetic security with NHibernate Membership Provider

zetetic has a pretty sweet encryption library that includes support for bcrypt2. It looks like it should be straightforward enough to incorporate into an ASP.NET Membership Provider (in fact, instructions for the default provider can be found here). I am using an NHibernate Membership Provider (found here) which seems to hard code the SHA1 hash format in it's EncodePassword function. My question is, how should this be adapted to work with BCrypt2 (specifically Zetetic's wrapper). This is something I greatly fear getting wrong, and I'm reluctant to take a stab at it myself lest it should "work" but have some hidden flaw that I am not qualified to find.
private string EncodePassword(string password)
{
string encodedPassword = password;
switch (PasswordFormat)
{
case MembershipPasswordFormat.Clear:
break;
case MembershipPasswordFormat.Encrypted:
encodedPassword =
Convert.ToBase64String(EncryptPassword(Encoding.Unicode.GetBytes(password)));
break;
case MembershipPasswordFormat.Hashed:
HMACSHA1 hash = new HMACSHA1();
hash.Key = HexToByte(_machineKey.ValidationKey);
encodedPassword =
Convert.ToBase64String(hash.ComputeHash(Encoding.Unicode.GetBytes(password)));
break;
default:
throw new ProviderException("Unsupported password format.");
}
return encodedPassword;
}
Are you modifying the NHibernate-based membership provider, or stuck using it out of the box? If the latter, it doesn't look like there's any extensibility.
The ASP.NET SqlMembershipProvider works by accepting the name of a hash algorithm, conjures an instance via HashAlgorithm.Create(name) and then behaves a little differently if the algorithm type turns out to be a KeyedHashAlgorithm or regular (non-keyed) HashAlgorithm. The Zetetic.Security package is just providing a little bit of glue to make BCrypt and PBKDF2 compatible with that model.
The sample code from NHMembershipProvider can't take advantage of that because it's very directly relying on HMACSHA1. I'd note that HMACSHA1 is not a secure algorithm for this purpose, nor is using a static salt for all users acceptable (it's scarcely better than no salt). The app ValidationKey and HMACSHA1 are meant for message integrity only.
Here's a sample:
public class HashDemo
{
private static readonly RNGCryptoServiceProvider s_rng = new RNGCryptoServiceProvider();
public string HashPassword(string pwd, string hashName)
{
var alg = HashAlgorithm.Create(hashName);
if (alg == null)
throw new ArgumentException("Invalid hash name", "hashName");
byte[] tohash = System.Text.Encoding.UTF8.GetBytes(pwd);
var ka = alg as KeyedHashAlgorithm;
if (ka != null)
{
if (ka.Key == null || ka.Key.Length == 0)
{
byte[] key = new byte[20];
s_rng.GetBytes(key);
ka.Key = key;
}
else
{
s_rng.GetBytes(ka.Key);
}
// TODO: return base64(ka.Key || alg.ComputeHash(tohash))
}
else
{
var salt = new byte[20];
s_rng.GetBytes(salt);
using (var ms = new System.IO.MemoryStream(salt))
{
ms.Write(tohash, 0, tohash.Length);
tohash = ms.ToArray();
}
// TODO: return base64(salt || alg.ComputeHash(tohash))
}
}
}

Resources