Redis implements three ways to share unique counts

A single count is one of the 10 common features of a website system. For example, a website needs to count the number of visitors per day. Counting problems are common, but can be complex to solve: 1) the amount of counting that needs to be done can be large, such as a large site with millions of people visiting it every day and a large amount of data; 2 is the dimension where you usually want to extend the count, such as UV per day and UV per week or month, which can complicate the calculation of 10 points.

In a relational database storage system, the only way to achieve a 1 count is select count(distinct) < item_id > ), it is 10 minutes simple, but if the data is large, this statement is slow to execute. Another problem with relational databases is that inserting data doesn't perform as well.

Redis is handy for solving this kind of counting problem, is faster than relational databases, consumes fewer resources, and even offers three different approaches.

1. Based on set

set of Redis is used to save the data set with only 1, through which it can quickly determine whether a certain element exists in the set, can also quickly calculate the number of elements in a certain set, and can be merged into a new set. The commands involved are as follows:

SISMEMBER key member  #  judge  member  If there is a 

SADD key member  #  Add to the set  member

SCARD key   #  Gets the number of collection elements  

The set method is simple, efficient, accurate, widely applicable, and easy to understand. Its disadvantage is that it consumes a lot of resources (of course, it is much less than a relational database). If the number of elements is large (such as counting billions), it consumes a lot of memory.

2. Based on bit

bit of Redis can be used to count highly compressed memory than set, which stores information about whether an element exists or not by 1 bit 1 or 0. For example, the visitor count of only 1 on a website can be user_id as the offset offset of bit. Setting user_id to 1 means that there is access. The space of 1 MB can store the 1-day visit count of more than 8 million users. The commands involved are as follows:

SETBIT key offset value  #  Setting bit information 

GETBIT key offset        #  Fetch bit information 

BITCOUNT key [start end] #  count 

BITOP operation destkey key [key ...]  #  The bitmap to merge  

The bit method USES much less space than set, but it requires elements to be easily mapped to bit offsets, which is much narrower. In addition, the amount of space it consumes depends on the maximum offset, which is independent of the count.

3. Based on HyperLogLog

Can realize accurate only 1 count of large amount of data is difficult, but if only approximate, there are many efficient algorithms in computer science, including HyperLogLog Counting is one of the very famous algorithm, it can be only used about 12 k memory, hundreds of millions of only 1 count, and error control in one percent. The commands involved are as follows:

PFADD key element [element ...]  #  Add elements 

PFCOUNT key [key ...]   #  count 

This method of counting is really amazing, and I haven't fully figured it out. If you are interested, you can further study relevant articles.

Each of the three unique counting methods provided by redis has its advantages and disadvantages, which can fully meet the counting requirements in different situations.