smallseo.info

jedis

A blazingly small and sane redis java client

how to store an image to redis using java / spring

I'm using redis and spring framework on my image upload server. I need to store the images to redis. I have found the following question but it was for python.
how to store a image into redis using python / PIL

I'm not sure if it's the best way but I would like to know how to do it in java (preferably using spring framework). I'm using spring-data-redis which uses jedis.

I would like to know if it is a good strategy to store images in redis.


Source: (StackOverflow)

single command to intersect redis sorted set by ranges

redis:

redis>zadd zsetA 1 'A'
redis>zadd zsetA 2 'B'
redis>zadd zsetA 3 'C'
redis>zadd zsetA 5 'E'
redis>zadd zsetB 1 'A'
redis>zadd zsetB 2 'B'
redis>zadd zsetB 3 'C'
redis>zadd zsetB 4 'D'

opearate:

a=redis.ZRANGEBYSCORE 'zsetA',1,3
b=redis.ZRANGEBYSCORE 'zsetB',2,4
result=a∩b

how to quick get 'result' in sigle command?


Source: (StackOverflow)

What is practical approach for fan-out-on-write in redis?

I have a news site that have a lot of topics. There might be millions of users following topics. I maintain a sortedset for each user to load news belonging to topics they are following. When an article is added or updated, I will write this article to affected users' lists. Specifically, pseudo code as follows:

if a article is added/updated
  get all topics that the article belong (each article may belong to many topics)
    for each topic: get all topic followers
      update_user_news_list(userId, articleId)

This is the java code with jedis:

static final int LIMIT_BATCH = 1000;
static void addToUserHomeFeed(int index, Jedis jd) {
        int range_limit = index + LIMIT_BATCH - 1;
        Set<String> list = jd.zrange("Follower:Topic:Id", index, range_limit); // get list of followers
        if (list.isEmpty())  return;
        Iterator<String> it = list.iterator();
        while (it.hasNext()) {
           // update user list
        }
        addToUserHomeFeed(range_limit + 1, jd);
}

The problem is, my site currently has nearly 1 million users, some popular topics followed by around 800000 users and sometimes the system produces "buffer overflow" errors. Am I doing something wrong or there are better approaches? I use redis 2.4


Source: (StackOverflow)

How to get multiple list values in one single call in RedisTemplate of Jedis Client

I am using RedisTemplate to get and store data as a list. When I am storing data - I store it as

redisTemplate.opsForList().rightPush("key1", "value11");
redisTemplate.opsForList().rightPush("key1", "value12");
redisTemplate.opsForList().rightPush("key2", "value21");
redisTemplate.opsForList().rightPush("key2", "value22");

Now I want to get the list values for both the keys in one single call I can get individually by

redisTemplate.opsForList().range("key1", 0, -1);
redisTemplate.opsForList().range("key2", 0, -1);

But is there a way to use multi get with list. If the values are of type string, I am able to use multisite, but I dont see any api with list.


Source: (StackOverflow)

JedisConnectionException: java.net.SocketTimeoutException: connect timed out

I'm using jedis for simple key-value data store... My code is as follows

private static final String HOST = "50.30.35.9"; 
private static final int PORT = 2863;    
Jedis jedis = new Jedis(HOST, PORT); 

  try { 
      jedis.set("foo", "bar"); 
      jedis.get("foo"); 
  } catch (Exception e) { 
      System.err.println("Unable to connect to Redis"); 
      e.printStackTrace(); 
  } 

When i test redis server with jedis client, i got the following exception.

 redis.clients.jedis.exceptions.JedisConnectionException: 
 java.net.SocketTimeoutException: connect timed out 
         at redis.clients.jedis.Connection.connect(Connection.java:124) 
         at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:  54) 
         at redis.clients.jedis.Connection.sendCommand(Connection.java:  77) 
         at redis.clients.jedis.BinaryClient.set(BinaryClient.java:71) 
         at redis.clients.jedis.Client.set(Client.java:21) 
         at redis.clients.jedis.Jedis.set(Jedis.java:48)

Can someone please help?


Source: (StackOverflow)

example retrieving redis values as stream in java

I have a redis key/value store holding blobs (size in the tens of MB), and the jedis client I am using in my java application returns a byte array from the jedis connection's get method. Currently, I have to wrap the result in a stream in order to process the bytes. Are there any alternatives that would allow me to stream the result directly? other clients or ways to use Jedsi? thanks for any advice.


Source: (StackOverflow)

What's the most efficient way to create a medium to large list/set/zset/hash in redis?

Using redis, there are a number of commands to retrieve entire data structures (LRANGE for lists, SMEMBERS for sets, ZRANGE for sorted sets, and HGETALL for hashes).

Only the hash has a method (HMSET) to insert multiple items with a single command.

All of the examples I've seen have shown just adding a single item at a time to a list (through RPUSH or LPUSH) or a set (through SADD/ZADD).

The more specific problem that I want to solve is to create lists and sorted sets containing database ids, these lists are unique to each user and contain a few hundred to a few thousand ids.

They are normally gathered from a database query, massaged a little bit in memory and then stored in redis for either paginating through (lists) or doing set based operations on to retrieve subsets (sets and sorted sets).

Currently, I'm iterating through the list and calling the appropriate add method for each element. This has the drawbacks of making multiple requests over the wire and repeating the key every time.

redis> RPUSH employee:ids 1000
(integer) 1
redis> RPUSH employee:ids 1001
(integer) 2
redis> RPUSH employee:ids 1002
(integer) 3
redis> RPUSH employee:ids 1003
(integer) 4
redis> del employee:ids
(integer) 1

I'm thinking that using a transaction with MULTI and EXEC could help to turn it into a single request, but that wouldn't help with the repeated key.

redis> MULTI
OK
redis> RPUSH employee:ids 1000
QUEUED
redis> RPUSH employee:ids 1001
QUEUED
redis> RPUSH employee:ids 1002
QUEUED
redis> RPUSH employee:ids 1003
QUEUED
redis> RPUSH employee:ids 1004
QUEUED
redis> EXEC
1. (integer) 1
2. (integer) 2
3. (integer) 3
4. (integer) 4
5. (integer) 5

Is there something that I'm missing that would let me add elements to lists/sets in a single command, or without repeating the key every time?

I'm also using the jedis client library if that matters (or if there's another library that I can use from the JVM that'd be better).


Source: (StackOverflow)

Redis performance on AWS EC2 Micro Instance

I have made a funny observation on my Redis instance deployed on my AWS EC2 Micro instance ( Test environment)

I was measuring the execution times of various operations that have to hit Redis. To summarise, the execution times ( average) are shown below:

Jedis -> Redis Connection is 63 milliseconds
Read of top Element in a list using lrange(<listname>,0,1) is 44 milliseconds
Read of entire Elements of set is 5ms
Iteration over entire Set space is 60ms( Set space  approx 130 elements)
Iteration over subset of elements of set is 5ms ( Subset element size is 5)

Now what is worrying me are the first 2 operations ( the connection and the extraction of the top element in a list).

For the connection, the code is shown below:

 Jedis redis= new Jedis("localhost");

And for the extraction of the top element in the list:

 String currentDate = redis.lrange(holderDate,0,1).get(0);

Now from the Redis lrange Command documentation:

Time complexity: O(S+N) where S is the start offset and N is the number of elements in the specified range.

Now from my code S would be 0 and N would be 1.

My question then is: What is causing these execution times for these somewhat trivial operations.

Are there characteristics of the EC2 Micro instance that would adversely affect the performance of these operations.

Some key information on the Redis deployment:

redis_version:2.4.10
used_memory:2869280
used_memory_human:2.74M
used_memory_rss:4231168
used_memory_peak:2869480
used_memory_peak_human:2.74M
mem_fragmentation_ratio:1.47

Thanks in advance.


Source: (StackOverflow)

jedis pubsub and timeouts: how to listen infinitely as subscriber?

I'm struggling with the concept of creating a Jedis-client which listens infinitely as a subscriber to a Redis pubsub channel and handles messages when they come in.

My problem is that after a while of inactivity the server stops responding silently. I think this is due to a timeout occurring on the Jedis-client I subscribe with.

Would this likely indeed be the case? If so, is there a way to configure this particular Jedis-client to not timeout? (While other Jedispools aren't affected with some globally set timeout) Alternatively, is there another (best practice) way of what I'm trying to achieve?

This is my code, (modified/ stripped for display) :

executed during web-server startup:

new Thread(AkkaStarter2.getSingleton()).start();

AkkaStarter2.java

   private Jedis sub;
   private AkkaListener akkaListener;

   public static AkkaStarter2 getSingleton(){
      if(singleton==null){
        singleton = new AkkaStarter2();
      }
      return singleton;
    }

    private AkkaStarter2(){
      sub = new Jedis(REDISHOST, REDISPORT);
      akkaListener = new AkkaListener();
    }

    public void run() {
      //blocking
      sub.psubscribe(akkaListener, AKKAPREFIX + "*");
    }

    class AkkaListener extends JedisPubSub {
        ....
        public void onPMessage(String pattern, String akkaChannel,String jsonSer) {
          ...
        }
    }

Thanks.


Source: (StackOverflow)

How to Convert SQL table into Redis Data

Hi I am new to redis and want some help over here. I am using java and sql server 2008 and redis server. To interact with redis I am using jedis api for java. I know that redis is used to store key value based things. Every key has values.

Problem Background:

I have a table names "user" which stores data like id, name, email, age, country. This is schema of sql table. Now this table have some rows(means some data as well). Now here my primary key is id and its just for DB use Its of no use for me in application.

Now in sql I can insert new row, can update a row, can search for any user, can delete a user.

I want to store this tables data into redis. Then I want to perform similar operations on redis as well, like search, insert, delete. But if I have a good design on "Storing this info in DB and Redis" then these operations will be carried out simply. Remember I can have multiple tables as well. So should store data in redis on basis of table.

My Problem

Any design or info you can advise me that how I can convert DB data to Redis and perform all operations. I am asking this because I know Facebook is also using redis to store data. Then how they are storing data.

Any help would be very appreciative.


Source: (StackOverflow)

Does Jedis support async operations

I am using Jedis (java client) to commmunicate with Redis server. I have 3 Redis instances running on three different nodes. I want to "get" (read) some records from 3 Redis instances. I want to issue these "gets" (reads) in parallel, and then do some processing on the received data and form a final output.

What is the best way to do this in java?

One of the way is to create 3 threads and isssue "get" (read) in each of them (synchronously). Wait for all 3 commands to complete and then combine the result.

Does Jedis have a mechanism for issuing 3 "gets" (any command for that matter) asynchronously, with a callback feature?


I have 3 different Redis instances. So do you suggest to use "ShardedJedisPipeline" (jedis/tests/ShardedJedisPipelineTest.java) for interacting with these Redis instances in parallel?

Normal Jedis Pipeline (jedis/tests/PipeliningTest.java), just sends multiple commands to single Redis instance so they are executed one after other on Redis server (and all responses available at the end).

So I assume I have to use "ShardedJedisPipeline". But there are two limitations to this: 1. I want to execute Lua script i.e. "eval" on 3 Redis instances in parallel. 2. I dont want sharding (some hash algorithm used by Jedis) to distribute data or on its own (using its algorithm) read data from instances. We have a different strategy for distributing data. So I want to be able to specify, a record should be stored in which redis instance and accordingly from where it should be read. keyTags seems to provide this mechanism but not sure how to use this with "eval".


Source: (StackOverflow)

Response Object in Jedis - throws ClassCastException

Using Response Object in Jedis, throws ClassCastException . I am not able to get any value from Redis when I use pipeline. Please help. I am using Jedis 2.1.0

public class JedisPipeline {    
    public static void main(String args[]){
        final JedisPool pool = new JedisPool(new JedisPoolConfig(), "127.0.0.1", 6379);
        Jedis jedis = pool.getResource();
        Pipeline pipeline = jedis.pipelined();
        pipeline.multi();
        HashMap<String,String> map = new HashMap<String,String>();
        map.put("50", "50");
        pipeline.hmset("Id",map);
        Response <Long> incr = pipeline.hincrBy("Id", "100", 100);
        Response<Map<String,String>> map1 = pipeline.hgetAll("Id");
        pipeline.exec();
        List<Object> results = pipeline.syncAndReturnAll();
        System.out.println(results);
        System.out.println( incr.get());
        System.out.println( map1.get());
        pool.returnResource(jedis);
        pool.destroy();
    }
}

Exception in thread "main" java.lang.ClassCastException: [B cannot be cast to java.lang.Long
    at redis.clients.jedis.BuilderFactory$4.build(BuilderFactory.java:45)
    at redis.clients.jedis.BuilderFactory$4.build(BuilderFactory.java:48)
    at redis.clients.jedis.Response.get(Response.java:27)
    at redis.clients.jedis.Pipeline.syncAndReturnAll(Pipeline.java:42)
    at com.work.jedis.JedisPipeline.main(JedisPipeline.java:28)

Source: (StackOverflow)

Jedis - When to use returnBrokenResource()

When exactly we should use this method. On JedisConnectionException, JedisDataException or for any JedisException. There is no good API documentation for Jedis to my knowledge.

try {
    Jedis jedis = JedisFactory.getInstance();
    Pipeline pipe = jedis.pipelined();
    Response<Set<Tuple>> idWithScore = pipe.zrangeWithScores(cachekey, from, to);
    **// some statement which may cause some other exception**
    Response<String> val = pipe.get(somekey);
    pipe.exec();
    pipe.sync();
}catch (JedisConnectionException e) {
    JedisFactory.returnBrokenResource(jedis);
}catch(Exception e){
    **// What API I should use here?, how to find whether to use returnBrokenResource(jedis) or returnResource(jedis)**
}finally{
    JedisFactory.returnResource(jedis);
}

Source: (StackOverflow)

Configure Jedis timeout

I'm having problems completing an .hgetall, here's what I've tried:

    Jedis jedis = new Jedis(REDIS_MASTER_NODE);
    jedis.connect();
    jedis.configSet("timeout", "30");

    Map<String, String> alreadyStored = jedis.hgetAll(redisTargetHash);

and here's what I get:

Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
    at redis.clients.jedis.Protocol.process(Protocol.java:79)
    at redis.clients.jedis.Protocol.read(Protocol.java:131)
    at redis.clients.jedis.Connection.getBinaryMultiBulkReply(Connection.java:199)
    at redis.clients.jedis.Jedis.hgetAll(Jedis.java:851)

Ok,

Jedis jedis = new Jedis(REDIS_MASTER_NODE, 6379, 1800);

did it.


Source: (StackOverflow)

Could not get a resource from the pool(SocketTimeoutException:)

I'm running multiple worker threads(around 10) to access the data from the redis Q.
For the i'm using infinte timeout for Jedis Client.

Jedis jedis = pool.getResource();
jedis.getClient().setTimeoutInfinite();  

Still i'm getting the error "Could not get a resource from the pool". The stacktrace is given below.

redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:22)
at Workers.Worker1.met1(Worker1.java:124)
at Workers.Worker1.work(Worker1.java:108)
at org.gearman.impl.worker.WorkerConnectionController$3.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)  

Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
at redis.clients.jedis.Connection.connect(Connection.java:124)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:54)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1657)
at redis.clients.jedis.JedisPool$JedisFactory.makeObject(JedisPool.java:63)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1188)
at redis.clients.util.Pool.getResource(Pool.java:20)
... 6 more  

Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at redis.clients.jedis.Connection.connect(Connection.java:119)
... 11 more

Source: (StackOverflow)