Summary:What kind of system is considered a high concurrent system? Today, let’s decipher the architecture of a typical seckill system in a high-concurrency business scenario.

This article is shared from the HUAWEI CLOUD community “[High Concurrency]Decryption of Seckill System Architecture, Not All Seckills Are Seckills (Upgraded Version)! ! “, Author: Glacier.

What kind of system is considered a high concurrent system? Today, let’s decipher the architecture of a typical seckill system in a high-concurrency business scenario.

E-commerce system architecture

In the field of e-commerce, there are typical flash sales scenarios, so what is a flash sale scenario? To put it simply, the number of buyers of a product is far greater than the inventory of this product, and this product will be sold out in a short period of time. For example, business scenarios such as the annual 618, Double 11 promotions, and Xiaomi new product promotions are typical flash sales scenarios.

We can simplify the architecture of the e-commerce system as shown in the figure below.

As shown in the figure, we can simply divide the core layer of the e-commerce system into: load balancing layer, application layer and persistence layer. Next, we estimate the concurrency of each layer.

• If the load balancing layer uses high-performance Nginx, we can estimate the maximum concurrency of Nginx to be: 10W+, and the unit here is 10,000.
• Assume that we use Tomcat at the application layer, and the maximum concurrency of Tomcat can be estimated to be about 800, and the unit here is hundreds.
• Assuming that the cache of the persistence layer uses Redis and the database uses MySQL, the maximum concurrency of MySQL can be estimated at about 1000, in units of thousands. The maximum concurrency of Redis can be estimated to be about 5W, in units of tens of thousands.

Therefore, the concurrency of the load balancing layer, the application layer, and the persistence layer are different. So, in order to improve the overall concurrency and caching of the system, what solutions can we usually adopt?

(1) System expansion

System expansion includes vertical expansion and horizontal expansion, adding equipment and machine configuration, which is effective in most scenarios.

(2) Cache

Local cache or centralized cache to reduce network IO and read data based on memory. Works in most scenarios.

(3) Read and write separation

Use read-write separation, divide and conquer, and increase the parallel processing capability of the machine.

Features of the seckill system

For the seckill system, we can start frombusiness and technologyTwo angles to explain some characteristics of its own existence.

The business characteristics of seckill system

Here, we can use the 12306 website as an example. During the annual Spring Festival travel season, the number of visits to the 12306 website is very large, but the usual visits to the website are relatively gentle. There is a sudden sudden increase.

Another example is Xiaomi’s seckill system, which starts selling products at 10:00 a.m. The number of visits before 10:00 a.m. is relatively flat, and at 10:00 a.m., there will also be a sudden increase in concurrent traffic.

Therefore, we can use the following figure to represent the traffic and concurrency of the seckill system.

It can be seen from the figure that the concurrency of the seckill system has the characteristics of an instantaneous peak, which is also called the traffic spike phenomenon.

We can summarize the characteristics of the seckill system as follows.

(1) Time-limited, limited, price-limited

It is carried out within the specified time; the number of goods in the seckill activity is limited; the price of the commodity will be much lower than the original price, that is to say, in the seckill activity, the commodity will be sold at a far lower price than the original price.

For example, the duration of the seckill activity is limited to 10:00 am to 10:30 am on a certain day, and the number of products is only 100,000 until they are sold out, and the price of the product is very low, such as: 1 yuan purchase and other business scenarios.

Time limit, limit and price limit can exist alone or in combination.

(2) Activity warm-up

The event needs to be configured in advance; before the event starts, users can view the relevant information of the event; before the seckill event starts, vigorously publicize the event.

(3) Short duration

The number of people buying is huge; items sell out quickly.

In terms of system traffic presentation, there will be a spike phenomenon. At this time, the concurrent visits are very high. In most flash sales scenarios, the products will be sold out in a very short time.

The technical characteristics of the seckill system

We can summarize the technical features of the seckill system as follows.

(1) The instantaneous concurrency is very high

A large number of users will rush to buy goods at the same time; the instantaneous peak value is very high.

(2) Read more and write less

The number of visits to product pages in the system is huge; the number of products that can be purchased is very small; the number of inventory query visits is far greater than the number of purchases of products.

Some current limiting measures are often added to the product page. For example, the product page of the early seckill system will add a verification code to smooth the front-end access traffic to the system. The product details page of the recent seckill system will prompt the user to log in to the system when the user opens the page. . These are some measures to limit the access to the system.

(3) The process is simple

The business process of the seckill system is generally relatively simple; generally speaking, the business process of the seckill system can be summarized as: placing an order and reducing inventory.

Spike three stages

Usually, from the beginning to the end of the seckill, there are three stages:

• preparation stage: This stage is also called the system warm-up stage. At this time, the business data of the seckill system will be preheated in advance. Often at this time, users will constantly refresh the seckill page to check whether the seckill activity has started. To a certain extent, some data can be stored in Redis for warm-up by users constantly refreshing the page.
• spike stage: This stage is mainly the process of seckill activities, which will generate instantaneous high concurrent traffic, which will cause a huge impact on system resources. Therefore, system protection must be done well during the seckill stage.
• Settlement stage: Complete the data processing work after the seckill, such as data consistency problem processing, abnormal situation processing, product return processing, etc.

For such a system with large traffic in a short period of time, it is not suitable to use system expansion, because even if the system is expanded, the expanded system will be used in a short period of time. Most of the time, the system It can be accessed normally without expansion. So, what solutions can we take to improve the system’s spike performance?

Spike system solution

According to the characteristics of the seckill system, we can take the following measures to improve the performance of the system.

(1) Asynchronous decoupling

The overall process is disassembled, and the core process is controlled through queues.

(2) Current limiting and anti brushing

Control the overall traffic of the website, increase the threshold of requests, and avoid exhaustion of system resources.

(3) Resource control

Control the resource scheduling in the overall process to maximize strengths and avoid weaknesses.

Because the concurrency that the application layer can carry is much less than the concurrency of the cache. Therefore, in a high-concurrency system, we can directly use OpenResty to access the cache from the load balancing layer, avoiding the performance loss of calling the application layer.everyone can comehttps://openresty.org/cn/Come learn more about OpenResty.At the same time, since the number of products in the seckill system is relatively small, we can also use dynamic rendering technology and CDN technology to speed up website access performance.

If the concurrency is too high at the beginning of the seckill activity, we can put the user’s request into the queue for processing, and pop up the queuing page for the user.

Note: The picture is from Meizu

Sequence diagram of seckill system

Many seckill systems and solutions to the seckill system on the Internet are not real seckill systems. What they use is only a synchronous request processing scheme. Once the concurrency really increases, the performance of their so-called seckill system will drop sharply. Let’s first take a look at the timing diagram of the seckill system when placing orders synchronously.

Synchronize the order process

1. The user initiates a seckill request

In the synchronous ordering process, first, the user initiates a seckill request. The mall service needs to execute the following processes in order to process the seckill request.

(1) Identify whether the verification code is correct

The mall service judges whether the verification code submitted by the user when initiating a seckill request is correct.

(2) Determine whether the activity has ended

Verify whether the current Lightning Deal has ended.

(3) Verify whether the access request is in the blacklist

In the field of e-commerce, there is a lot of malicious competition, that is to say, other merchants may maliciously request to kill the system through improper means, occupying a large amount of bandwidth and other system resources of the system. At this time, it is necessary to use a risk control system to implement a blacklist mechanism. For simplicity, you can also use the interceptor to count the access frequency to implement the blacklist mechanism.

(4) Verify that the real inventory is sufficient

The system needs to verify whether the real inventory of the product is sufficient and whether it can support the product inventory of this flash sale activity.

(5) Deduct the inventory in the cache

In the seckill business, information such as commodity inventory is often stored in the cache. At this time, it is also necessary to verify whether the commodity inventory used by the seckill activity is sufficient, and to deduct the quantity of commodity inventory used in the seckill activity.

(6) Calculate the price of seckill

Since there is a difference between the flash sale price of the commodity and the real price of the commodity in the flash sale activity, it is necessary to calculate the flash sale price of the commodity.

Note: If the business involved in the system is more complicated in the seckill scenario, more business operations will be involved. Here, I just list some common business operations.

2. Submit the order

(1) Order entry

Save the order information submitted by the user to the database.

(2) Deduction of real inventory

After the order is placed in the warehouse, the quantity of the successfully placed product needs to be deducted from the real inventory of the product.

If we use the above process to develop a seckill system, when the user initiates a seckill request, since each business process of the system is executed serially, the performance of the overall system will not be too high. When the concurrency is too high, we will The following queuing page pops up for the user to prompt the user to wait.

Note: The picture is from Meizu

The queuing time at this time may be 15 seconds, or 30 seconds, or even longer. There is a problem here: the connection between the client and the server will not be released during the period from when the user initiates the seckill request to when the server returns the result, which will occupy a large amount of server resources.

Many articles on the Internet that introduce how to implement the seckill system use this method. So, can this method be used as a seckill system? The answer is that it can be done, but the amount of concurrency supported by this method is not too high. At this time, some netizens may ask: This is how our company uses the seckill system! I have been using it since it went online, no problem! What I want to say is: using the method of synchronous order placement can indeed be used as a seckill system, but the performance of synchronous order placement will not be too high. The reason why your company does not have any major problems with the seckill system by placing orders synchronously is because the concurrency of your seckill system has not reached a certain level, that is to say, the concurrency of your seckill system is actually not high. high.

Therefore, many so-called seckill systems have seckill services, but they cannot be called real seckill systems because they use a synchronous order process, which limits the concurrent traffic of the system. The reason why there are no major problems after going online is because the concurrency of the system is not high enough to overwhelm the entire system.

If 12306, Taobao, Tmall, JD.com, Xiaomi and other large shopping malls operate their seckill systems in this way, then their systems will be killed sooner or later, and it’s no wonder their system engineers are not fired! Therefore, in the seckill system, this scheme of synchronously processing the business process of placing an order is not advisable.

The above is the entire process operation of synchronous ordering. If the ordering process is more complicated, more business operations will be involved.

Asynchronous order process

Since the seckill system that synchronizes the order process cannot be called a real seckill system, we need to adopt an asynchronous order process. The asynchronous order process will not limit the high concurrent traffic of the system.

1. The user initiates a seckill request

After the user initiates a seckill request, the mall service will go through the following business process.

(1) Check whether the verification code is correct

When the user initiates a seckill request, the verification code will be sent together, and the system will check whether the verification code is valid and correct.

(2) Whether to limit current

The system will judge whether the user’s request is limited or not. Here, we can judge by judging the length of the message queue. Because we put the user’s request in the message queue, and the user’s request is accumulated in the message queue, we can judge whether it is necessary to limit the flow of the user’s request according to the number of pending requests in the current message queue.

For example, in the seckill activity, we sell 1,000 items, and there are 1,000 requests in the message queue at this time. If there are still users who initiate seckill requests in the future, we can stop processing the subsequent requests and directly return the product to the user as sold finished prompt.

Therefore, after using current limiting, we can process user requests faster and release connection resources.

(3) Send MQ

After the user’s seckill request passes the previous verification, we can send the user’s request parameters and other information to MQ for asynchronous processing, and at the same time, respond to the user with the result information. In the mall service, there will be a dedicated asynchronous task processing module to consume requests in the message queue and process subsequent asynchronous processes.

When a user initiates a seckill request, the asynchronous order process handles fewer business operations than the synchronous order process. It sends subsequent operations to the asynchronous processing module through MQ for processing, and quickly returns the response result to the user to release the request connection.

2. Asynchronous processing

We can process the following operations of the order process asynchronously.

(1) Determine whether the activity has ended

(2) Determine whether this request is in the system blacklist. In order to prevent malicious competition from peers in the e-commerce field, a blacklist mechanism can be added to the system to put malicious requests into the system blacklist. It can be realized by using the interceptor to count the access frequency.

(3) Deduct the inventory quantity of flash sale products in the cache.

(4) Generate a seckill token, which is bound to the current user and the current seckill activity, and only the request that generates the seckill token is eligible for the seckill activity.

Here we introduce an asynchronous processing mechanism,In asynchronous processing, it is possible to control how many resources the system uses and how many threads are allocated to handle corresponding tasks.

3. Short polling query spike results

Here, a solution may be adopted in which the client short polls to inquire whether it is qualified for the flash sale. For example, the client can poll the requesting server every 3 seconds to check whether it is qualified for the seckill. Here, our processing on the server is to determine whether the current user has a seckill token. If the server generates a seckill token for the current user, the current user There is a spike qualification. Otherwise, continue to poll until the timeout or the server returns information such as that the product has been sold out or is not eligible for flash sales.

When short polling is used to query the results of the flash sale, we can also prompt the user to queue up on the page, but at this time the client will poll the server every few seconds to check the status of the flash sale qualification. Compared with the synchronous order process, There is no need to hold up the request connection for a long time.

at this time,Some netizens may ask: With the method of short polling query, will there be a state where the eligibility for the flash sale will not be queried until the timeout? The answer is: possible! Here, let’s imagine the real scene of seckill. Merchants participate in seckill activities not to make money in essence, but to increase the sales of products and the popularity of merchants, so as to attract more users to buy their products. Therefore, we don’t have to guarantee that users can 100% check whether they have the status of flashkill qualifications.

4. Instant Kill Settlement

(1) Verify the order Token

When the client submits the seckill settlement, it will submit the seckill token to the server, and the mall service will verify whether the current seckill token is valid.

(2) Add to the Lightning Shopping Cart

After the mall service verifies that the seckill Token is legal and valid, it will add the user’s seckill product to the seckill shopping cart.

5. Submit the order

(1) Order storage

Save the order information submitted by the user to the database.

(2) Delete Token

After the seckill product order is successfully stored, the seckill Token is deleted.

Here you can think about a question: Why do we only use asynchronous processing in the pink part of the asynchronous ordering process, but not take asynchronous peak-shaving and valley-filling measures in other parts?

This is because in the design of the asynchronous ordering process, whether it is in product design or interface design, we limit the flow of the user’s request when the user initiates a second kill request. It can be said that the flow limit operation of the system is Very forward. When the user initiates a seckill request, the traffic is limited, and the peak traffic of the system has been smoothly resolved. Going forward, the concurrent volume and system traffic of the system are not very high.

Therefore, when introducing the seckill system in many articles and posts on the Internet, it is said that asynchronous peak clipping is used to perform some current limiting operations when placing an order, which is all nonsense! Because the order placing operation is a relatively late operation in the process of the entire seckill system, the current limiting operation must be pre-processed, and it is useless to perform the current limiting operation in the subsequent process of the seckill business.

High concurrency “black technology” and winning tricks

Assume that we use Redis to implement caching in the seckill system, assuming that the concurrent reading and writing of Redis is about 50,000. The concurrency that our mall seckill business needs to support is around 1 million. If all the 1 million concurrency are entered into Redis, Redis is likely to hang up, so how do we solve this problem? Next, we will discuss this issue together.

In a high-concurrency seckill system, if Redis is used to cache data, the concurrent processing capability of Redis cache is the key, because many prefix operations need to access Redis. Asynchronous peak shaving is only a basic operation, and the key is to ensure the concurrent processing capability of Redis.

The key idea to solve this problem is: divide and conquer, separate the commodity inventory.

Darkness

When we store the inventory quantity of flash sale products in Redis, we can “segment” the stock of flash sale products to increase the concurrent reading and writing of Redis.

For example, the id of the original flash sale product is 10001, the inventory is 1000 pieces, and the storage in Redis is (10001, 1000). We divide the original inventory into 5 parts, and the inventory of each part is 200 pieces. At this time , the information we store in Redia is (10001_0, 200), (10001_1, 200), (10001_2, 200), (10001_3, 200), (10001_4, 200).

At this point, after we divide the inventory, each divided inventory is stored using the commodity id plus a digital identifier. In this way, when the Hash operation is performed on each Key that stores the commodity inventory, the Hash results obtained are different. Yes, this means that there is a high probability that the Key for storing commodity inventory is not in the same slot of Redis, which can improve the performance and concurrency of Redis processing requests.

After the inventory is divided, we also need to store a mapping relationship between the product id and the Key after the inventory is divided in Redis. At this time, the Key of the mapping relationship is the id of the product, which is 10001, and the Value is the Key for storing inventory information after the inventory is divided. , that is, 10001_0, 10001_1, 10001_2, 10001_3, 10001_4. In Redis we can use List to store these values.

When actually processing inventory information, we can first query from Redis all the keys corresponding to the split inventory corresponding to the seckill product, and at the same time use AtomicLong to record the current request quantity, and use the request quantity to match the corresponding seckill commodity queried from Redia The length of all the keys after the inventory is divided is modulo, and the results are 0, 1, 2, 3, 4. Then splice the product id in front to get the real inventory cache key. At this point, you can directly go to Redis to obtain the corresponding inventory information according to this Key.

graft

In high-concurrency business scenarios, we can directly use the Lua script library (OpenResty) to directly access the cache from the load balancing layer.

Here, let’s think about a scenario: If in the seckill business scenario, the products of the seckill are sold out in an instant. At this time, when the user initiates a seckill request again, if the system requests the services of the application layer from the load balancing layer, and then the services of the application layer access the cache and database, in fact, it is meaningless in essence, because the products have been sold out , and then verify layer by layer through the application layer of the system, it doesn’t make much sense anymore! ! The number of concurrent visits at the application layer is in units of hundreds, which will reduce the concurrency of the system to a certain extent.

To solve this problem, at this time,In the load balancing layer of the system, we can take out the user id, product id and seckill activity id carried by the user when sending the request, and directly access the inventory information in the cache through techniques such as Lua scripts. If the inventory of the flash sale product is less than or equal to 0, it will directly return the prompt message that the user’s product is sold out, without going through the layer-by-layer verification of the application layer. For this architecture, we can refer to the architecture diagram of the e-commerce system in this article (the first diagram at the beginning of the text).

Redis helps spike system

We can design a Hash data structure in Redis to support the deduction operation of commodity inventory, as shown below.

seckill:goodsStock:${goodsId}{
totalCount:200,
initStatus:0,
seckillCount:0
}

In the Hash data structure we designed, there are three very main attributes.

• totalCount: Indicates the total number of products participating in the flash sale. Before the flash sale starts, we need to load this value into the Redis cache in advance.
• initStatus: We design this value as a boolean value. Before the seckill starts, this value is 0, indicating that the seckill has not started. You can change this value to 1 through scheduled tasks or background operations, which means the seckill has started.
• seckillCount: Indicates the number of seckilled items. During the seckill process, the upper limit of this value is totalCount. When this value reaches totalCount, it means that the seckill has been completed.

We can use the following code snippet to cache the product data that will participate in the flash sale during the flash sale warm-up phase.

/**
 * @author binghe
 * @description 秒杀前构建商品缓存代码示例
 */
public class SeckillCacheBuilder{
 private static final String GOODS_CACHE = "seckill:goodsStock:"; 
 private String getCacheKey(String id) { 
 return GOODS_CACHE.concat(id);
 } 
 public void prepare(String id, int totalCount) { 
 String key = getCacheKey(id); 
 Map<String, Integer> goods = new HashMap<>(); 
 goods.put("totalCount", totalCount); 
 goods.put("initStatus", 0); 
 goods.put("seckillCount", 0); 
 redisTemplate.opsForHash().putAll(key, goods); 
 }
}

When seckill starts, we need to first judge whether the seckillCount value in the cache is less than the totalCount value in the code. If the seckillCount value is indeed less than the totalCount value, we can lock the inventory. In our program, these two steps are not actually atomic. If in a distributed environment, we operate Redis cache through multiple machines at the same time, synchronization problems will occur, which will cause serious consequences of “overselling”.

In the field of e-commerce, there is a professional term called “overselling”. As the name suggests: “Overselling” means that the number of products sold is more than the number of products in stock, which is a very serious problem in the field of e-commerce. So, how do we solve the “oversold” problem?

Lua script perfectly solves overselling problem

How do we solve the synchronization problem that occurs when multiple machines operate Redis at the same time? A better solution is to use Lua scripts. We can use Lua script to encapsulate the operation of deducting inventory in Redis into an atomic operation, so as to ensure the atomicity of the operation and solve the synchronization problem in a high-concurrency environment.

For example, we can write the following Lua script code to perform the inventory deduction operation in Redis.

local resultFlag = "0" 
local n = tonumber(ARGV[1]) 
local key = KEYS[1] 
local goodsInfo = redis.call("HMGET",key,"totalCount","seckillCount") 
local total = tonumber(goodsInfo[1]) 
local alloc = tonumber(goodsInfo[2]) 
if not total then 
 return resultFlag 
end 
if total >= alloc + n  then 
 local ret = redis.call("HINCRBY",key,"seckillCount",n) 
 return tostring(ret) 
end 
return resultFlag

We can use the following Java code to call the above Lua script.

public int secKill(String id, int number) { 
 String key = getCacheKey(id); 
 Object seckillCount = redisTemplate.execute(script, Arrays.asList(key), String.valueOf(number)); 
 return Integer.valueOf(seckillCount.toString()); 
}

In this way, we can ensure the atomicity of the operation when we execute the seckill activity, thereby effectively avoiding the data synchronization problem and effectively solving the “oversold” problem.

Click to follow and learn about Huawei Cloud’s fresh technologies for the first time~