Netty's object pool

brief introduction

This article talks about Netty Object pool technology of . The code in this article is based on 4.1 To analyze .
Different from the focus of memory management is , Object pool technology is mainly responsible for object recycling .
let me put it another way , Object pool technology aims at the recycling of objects , The subject of management is the object , But objects also need memory space to create , So in the process , Memory is just the carrier of objects .
Memory management technology is aimed at independent memory blocks , The subject of management is the object , But we need an object to represent the reference to the memory block , So that we can visit , So in the process , Object is actually the carrier of memory .
Because these two technologies are often used together , So before we start learning the following process , It is important to clarify the difference between the two .

Object pool ——Recycler

Recycler A class is an object pool , The key logic of object management is in this class .
Recycler Is an abstract generic class . Generic parameters represent the actual use of the scenario , The types of objects that need to be managed .
Although this class is declared abstract , But the main logic of object management is fixed ——Recycler Most methods are declared as final, Show that it doesn't want subclasses to modify the logic . And what's left for subclasses is just newObject() Method , When there are no cached objects in the pool , Used to create new objects （ Because the logic of creating objects may need to be defined by users themselves ）.

Start with attributes

Attributes are divided into two parts ：
One 、 It's a global configuration , It is usually used to provide the default disposal value without setting the initial value . These are static properties .

• DEFAULT_INITIAL_MAX_CAPACITY_PER_THREAD The default value is 4 * 1024
• DEFAULT_MAX_CAPACITY_PER_THREAD By default DEFAULT_INITIAL_MAX_CAPACITY_PER_THREAD Default value , namely 4 * 1024
• INITIAL_CAPACITY The default value is DEFAULT_MAX_CAPACITY_PER_THREAD or 256
• MAX_SHARED_CAPACITY_FACTOR The default value is 2
• MAX_DELAYED_QUEUES_PER_THREAD The default value is CPU Twice as many
• LINK_CAPACITY The default value is 16 LINK Size （LINK It's a node in the queue ,LINK Connect with each other , Form a line , meanwhile LINK Inside is another array , You can store multiple objects , The size of the array is determined by LINK_CAPACITY control ）
• RATIO The default value is 8
  The default values of these parameters can be adjusted by specific system parameters .

Two 、 It's the configuration of objects , If you're creating Recycler when , Constructor with the relevant configuration , Then these configurations override the default parameters .

• maxCapacityPerThread, The corresponding default value is... Above DEFAULT_MAX_CAPACITY_PER_THREAD, Represents the maximum capacity of each thread , It indicates that the Stack Maximum stack depth of (Stack Its functions are described below )
• maxSharedCapacityFactor The corresponding default value is... Above MAX_SHARED_CAPACITY_FACTOR,, Represents each thread maxCapacityPerThread And sharedCapacity The proportion of , namely sharedCapacity = maxCapacity / maxSharedCapacityFactor
• ratioMask And above RATIO relevant , Represents the recovery ratio , To control the frequency of recycling , Avoid recycling too fast
• maxDelayedQueuesPerThread, And above MAX_DELAYED_QUEUES_PER_THREAD relevant , Represents the maximum number of Queue The number of （Queue And its functions are also described in detail below ）

The four related properties that really affect the configuration of the object pool , The static properties above only provide default values for this configuration item .

Besides , There is also a member variable in the class FastThreadLocal<Stack<T>> threadLocal. understand jdk Our readers should know ThreadLocal Is used to store thread local variables , and FastThreadLocal and ThreadLocal The same effect , But the performance is optimized . From the generic parameters, we can see that what is stored at this time is Stack Class object .
Already a static variable DELAYED_RECYCLED, The same is FastThreadLocal, It's just that the type of preservation is Map, among Map Of Key yes Stack, and Value yes WeakOrderQueue. We'll see later that this variable holds one thread for another Stack Created WeakOrderQueue.

Several inner classes and their associations

Recycler There are still fewer attributes of , But there are several inner classes , Namely :

• Stack—— Used to store recycled objects
• WeakOrderQueue—— Store objects recycled by other threads
• DefaultHandle—— Object handle

Stack—— The stack used to store recycled objects

Stack Is the core class for storing and recycling objects . When recycling objects , The object is pushed into the stack Stack Kept in (push() The process ). And when the applicant is , The saved object will be popped up to the applicant through the stack (pop() The process ).
meanwhile , Each thread has its own Stack example （ You can read from the above FastThreadLocal<Statk<T>> To determine ）, Explain each thread Final They all recycle their own created objects and save them ( Note that Final , Other threads may be involved in helping recycle , And save it to WeakOrderQueue Middle transition ).

there Stack It's not used directly JDK Provided in java.util.Stack, because java.util.Stack Not having some of the extra features needed here . Instead, it relies directly on arrays to re implement .
To understand Stack The internal structure of ：

// Associated object pool Recycler object
final Recycler<T> parent;
// The thread has a reference to the stack ( Thread )
final WeakReference<Thread> threadRef;
/**** from Recycler Set the related properties of ******/
// Shared capacity
final AtomicInteger availableSharedCapacity;
// The number of queues
final int maxDelayedQueues;
// Maximum stack depth
private final int maxCapacity;
// Control the recycling ratio
private final int ratioMask;
// The array of dependencies at the bottom of the stack It's a handle ——DefaultHandle, Instead of a reference to a direct object
private DefaultHandle<?>[] elements;
// Stack size
private int size;
// Recycle count , coordination ratioMask You can decide whether to recycle this time
private int handleRecycleCount = -1; // Start with -1 so the first one will be recycled.
/********** WeakOrderQueue The chain formed *****************/
// Pointer to the current , The previous pointer ; Used to decide from what WeakOrderQueue Transfer objects to Stack in
private WeakOrderQueue cursor, prev; //cursor Record the current WeakOrderQueue The location of the linked list Because the linked list is the head insert So we need to cursor Mark
// The actual header of the linked list
private volatile WeakOrderQueue head; // Real chain header nodes Every time you create a new WeakQueue when It will be inserted into the linked list as the head node

From the above code, we can learn a few information ：

• Stack Internal use of arrays to store object handles (DefaultHandler), The maximum depth of the stack is the capacity of the array , from Recycler To determine the relevant properties of
• Every Stack It's all thread private ,Stack Has threads through threadRef Record
• Stack There's a WeakOrderQueue The linked list of , In addition to recording the header of the linked list (head) Outside , It also records the current cursor of the linked list (cursor), And labeled predecessor nodes (prev)

Stack For the time being, we'll analyze the code here , The following will be described in detail in the process of object recovery and application .

DefaultHandle—— Default object handle

DefualtHandle It's the interface Handle Default implementation of , The interface declares a method ——void recycle(Object object), That is, when an object is recycled , The collection process starts with the handle .

In the morning Netty In the version ,Recycler It directly provides an interface for recycling , But this interface has been abandoned , In its place is Handler.recycle The interface of . So you can hide Stack and Recycler Some of the details of .

DefaultHandle yes Handle Default implementation of , The internal structure is relatively simple .

 static final class DefaultHandle<T> implements Handle<T> {
// Record recycled id And whether it's recycled or not
private int lastRecycledId;
private int recycleId;
boolean hasBeenRecycled;
// Handle associated stack
private Stack<?> stack;
// Handle refers to the object
private Object value;
// Construction method And stack binding
DefaultHandle(Stack<?> stack) {
this.stack = stack;
}
// Recycling action , Objects on the stack
@Override
public void recycle(Object object) {
if (object != value) {
throw new IllegalArgumentException("object does not belong to handle");
}
// Pushing objects onto the stack
stack.push(this);
}
}

DefaultHandle The code is relatively simple , From the above code can also sum up a few points ：

• Handle through value Objects hold references to objects
• Handle and Stack Objects are interrelated ,Stack After assigning objects , The handle to the object is the same as Stack The binding , In this way, you can know from the handle which object it is Stack The distribution of , You can then infer which thread is responsible for creating

WeakOrderQueue—— Threads help reclaim the staging places of objects not created by this thread

On the whole WeakOrderQueue Is used to temporarily store recycled objects . So what kind of object will be WeakOrderQueue Temporary storage first , It's not stored directly in Stack What about China? ？
The answer is if the thread performing the recycle is not the creation thread of the object ( As mentioned earlier, the handle knows the associated Stack And threads ), Then this recycle will be temporarily stored in WeakOrderQueue In excess of .
The advantage of this is that it can reduce the competition between threads , Increase throughput .

From the inside ,WeakOrderQueue By Link The list of components , Can be Link Think of it as a node in a linked list .
Link Related codes ：

 static final class Link extends AtomicInteger {
//DefaultHandle Array of , Store the recycled handle
private final DefaultHandle<?>[] elements = new DefaultHandle[LINK_CAPACITY];
// Record read index , The rest is unread
private int readIndex;
// Point to next node , Form a linked list
Link next;
}

Link The head of a linked list is a special structure , There are two main functions , One is when it's recycled , adopt finalize() Implement the release operation , The other is when adding nodes , You need to make sure that your space is still available , Avoid going beyond maxSharedCapacity The limitation of .

WeakOrderQueue In addition to the two special properties described above , Other properties are relatively simple .

 // Dummy node
static final WeakOrderQueue DUMMY = new WeakOrderQueue();
// Head node
private final Head head;
// Tail node
private Link tail;
// pointer to another queue of delayed items for the same stack
// I've already described Stack There will be WeakOrderQueue The chain formed , Through this next Pointer in series
private WeakOrderQueue next;
// Associated threads there thread No stack The thread of , It is weakOrderQueue Thread in
private final WeakReference<Thread> owner;
//ID Number
private final int id = ID_GENERATOR.getAndIncrement();

To sum up ：

• WeakOrderQueue The internal is Link The list of components , Every Link There is one DefaultHandle Array of , To save objects
• WeakOrderQueue They form a list of each other , It means something Stack All under WeakOrderQueue

Related processes

In understanding Recycler And the structure of the internal main classes , We'll pass it again Recycler Recycling and application process , Deepen the role of variables and inner classes .

Object recycling process

As mentioned before , The recycling process of an object is from calling DefaultHandle.recycle() Method start . We take this as the entrance , Let's take a look at the object recycling process .

1. After recycling , The handle first validates the recycled object, the referenced object , And then it's internally related Stack Through the stack operation , Recovery object , namely stack.push(defaultHandle);
2. The specific stack process depends on whether the thread executing the recycle action should be stack The owners are divided into pushNow() and pushLater() Two processes
3. If the recycled thread A It's the time to stack The owner of the , Description is thread A Recycle your own created objects , Then through the pushNow() Recycle objects directly to Stack The internal array holds （ Of course , Also need to consider ratioMask And the capacity of the array , The former is used to control the frequency of recycling , Avoid recycling too fast ; The latter is used to control the maximum amount of recycling , Avoid recycling too much ）
4. If the recycled thread A It's not the time to stack The owner of the , Description is not object creation thread recycling （ We first call the object creation thread B）, Then it will enter pushLater() Try to hold the object to a specific WearOrderQueue in . If you look for specific WeakOrderQueue Well ？ First , Through the introduction of the type of FastThreadLocal The variable of DELAYED_RECYCLED, Get the recycle thread first A All created WeakOrderQueue, Get one Map object , Through stack Object to find threads A Is it for the sake of stack Created WeakOrderQueue. If you don't create , Try to create a WeakOrderQueue( But if it's already threaded A Created WeakOrderQueue Has reached the maximum number or should be Stack The maximum shared capacity of is not enough , Then no new WeakOrderQueue, It will not recycle the object . Besides , In the case of the former , Also in the map In order to Stack Associate a special dummy node DUMMY, Indicates that you will not try to create a new WeakOrderQueue). If you can build a new one WeakOrderQueue Or already have WeakOrderQueue, Then by WeakOrderQueue Temporary objects . Save the object in WeakOrderQueue Inside Link In the array of the end nodes of the linked list . If the tail node is full , It will create a new one Link node , And add it to the end of the list , Become the new tail node . Empathy , new Link Node creation also needs to consider whether the maximum shared capacity is exceeded avaliableSharedCapacity, If you exceed , Refuse to create a new Link node , The object is not recycled .

Object application process

The application process of the object is from Recycler.get() At the beginning , That is to get the object from the object pool . The process is as follows ：

1. Get thread associated Stack, As mentioned earlier, each thread has its own Stack To save objects . If the thread has not yet Stack, Through initValue() Create a Stack.Recycler Will be used to create Stack.
2. from Stack Try to pop up the object in (stack.pop()), At this point, if the object can be ejected , Explain the Stack Objects have been recycled before . If there is no recycled object , A new object is created .
3. Creating a new object is a two-step process , First, created by stack.newHandle() Create an object handle , second , By going through newObject(handle) Method to create an object , This is a Recycler Abstract method of , By the concrete object pool subclass according to the management object does not need to realize . Got DefaultHandle Will hold a reference to the object .

The process of creating new objects is relatively simple , The main thing is to understand from Stack The process of ejecting objects in the . We've learned that recycled objects may be stored in Stack The inner array and WeakOrderQueue in Link Array of two places , In fact, pop-up is also from here to find the object , And back to .
First , The stack process will first get the elements from the stack , If there are no elements in the stack at this time , So from WeakOrderQueue Move the temporary element to the stack . Then we get the elements from the end of the stack .

Recycler Relationships between related classes

Simple will Recycler The relationship between the inner classes draws a diagram . To help readers understand the relationship between a given class under different threads .

reflection

Netty It actually provides a very powerful object pool framework , With this framework, we can easily implement our own object pool requirements , Such as connection pool, etc .
Detailed source code comments can be seen for Github.