The idea and practice of coupon offer

Foreword: the recent development about coupons, but found that large amount of coupons, completely failed to keep pace with performance, library in 2 million coupons, send 1 card voucher need 5 minutes, then I start optimization, the final departure 1 ticket only need about 15 ms zhang, now the whole train of thought and post code, for everybody to 1 discussion and study.

Introduction to the

The main implementation of coupon promotion activities, first to create an activity, and then to create a coupon group, using the way of pre-processing coupon making, in the first version of the main implementation, the basic function of the business. Then in the branch implementation, the large number and high concurrency problem.

Branch 1.1

1: solve the problem of coupon code repetition, originally is the access to the database all the stamps, and then to compare whether repeat, if the library data volume millions will appear very slowly, and often there will be a system of vouchers for failure, etc., so this version abandon originally used random pattern, through twitter snowflakes algorithm to avoid only 1, but still reserve coupons prefixes and suffixes.

2: The original asynchronous thread is changed to thread pool, so that a large number of threads occupy the memory space during high concurrency.

3. The original coupon making is changed from for cycle mode to batch coupon making, and the allocation and insertion of coupons are adopted. The current number of batches is 5000.

4: Join message queue (using rabbitMQ) For a batch of failure to add, put the failed into the column, through the queue to repair, has reached high availability. Avoid reimporting the message queue back and forth with bulk coupons. Resolve the exception message rejection and return to the message queue. Configure 2 consumers to avoid one of the service exceptions

Branch 1.2

1: Join the operation log

Purpose: To track hot data and query logs to quickly track slow queries or slow operations in an application, laying the foundation for subsequent optimization

2: Add exception log

Purpose: To quickly get the thread exception problem, through the log data can be quickly modified

3: Adopt the technology to do through aop and rabbitmq middleware, so as to reduce the efficiency problem brought to the program by the logging problem

No optimization efficiency statistics

Database mysql was used

Data: 25 valid activities were added. There were two coupon groups under each activity, and the coupon group was 50,000. There are 2.5 million records in the coupon table

Business: 50 coupons will be sent to 1 member to meet these 25 activities.

Statistics: About 306s was consumed during the whole coupon issuing process after 10 statistics, and 6s was consumed each time the coupon was obtained. Performance bottlenecks are inevitable if multiple cycles occur

It takes about 0.5s to update the status of the coupon, from which we can see that our performance problem is mainly due to the coupon acquisition. That's why version 1.3 mainly addresses this issue programmatically

Branch 1.3

Purpose: To improve performance through program code and database optimization

Specific plan:

1: All coupons under the coupon acquisition group before are now changed to only 100 coupons at a time (according to the test statistics, the sending time of 50 coupons is 106s, and the time of each coupon acquisition is about 2s, with the overall performance improved by nearly 3 times)

2: Optimize sql and add composite index (statistics show that the total time of sending 50 coupons is 2.5s, and the time of each coupon acquisition is about 0.015s, with the overall performance improved by nearly 42 times)

3: Join the local cache (if the one-time coupons are put into map first, then the next time there are coupons, there is no need to get the coupons from the library. Statistics: 10 items, 50 coupons for each item

The efficiency without local cache is 7.5ES67en. The efficiency after local cache is added is about 5.5s. The overall performance is improved by 2s.)

Effect analysis:

4. Batch update is adopted to replace for cycle for issuing coupons (the performance of s is improved to about 4.8s from about 5.5 above).

Branch 1.4

Purpose: To issue coupons through asynchrony and message queuing

Specific plan:

1. Issuing coupons asynchronously can improve the utilization rate of cpu and ensure the stability through message queue, avoiding the occurrence of exceptions that lead to the successful issuing of coupons back to the front end. However, when issuing 500 coupons asynchronously, the efficiency of s is about 0.5 higher

2: Refactor the code once

Principle: change the large method to the small method, each small method to handle 1 business, such as get activity, then the responsibility of this method is to get activity, at the same time each small method as far as possible return value, so as to increase the readability of the code

Branch 1.5

1. redis was used as cache, and the valid activities of the day were taken, and the coupon group was activated, and the 500 coupons in the coupon group were stored in the cache.

2. Add a timed task to update the cache at 12 o 'clock every day (this time can be monitored through the hotspot data)

3. Statistical findings:

It takes only 2.7s to send 500 coupons after caching, which is 2.1s faster than the previous 4.8s, greatly improving performance

Conclusion: I will not paste the code, you can go to see by yourself. Interested friends can continue to study and develop on this basis. In version 1.6, the sub-library sub-table may be added, and the current version is dangdang ES107en-ES108en

The source address