Of Stack

Swift source parsing weak references

  • 2020-06-15 10:21:21
  • OfStack

Preface:

There has been a lot of community analysis on the implementation of Objective-ES4en weak mechanisms, but in the long time since Swift was published, there has been very little analysis on ABI 1, which seems to be an area where many iOS developers have not been involved... This paper analyzes how Swift implements weak mechanism from the source level.

Without further ado, let's take a look at the details

The preparatory work

Due to the large amount of Swift source code, we strongly recommend repo clone down, combined with the source code 1 to see this article. 


$ git clone https://github.com/apple/swift.git

Swift USES CMake as the build tool throughout the project. If you want to use Xcode to open it, you need to install LLVM first and then generate Xcode project with ES29en-ES30en.

Here we are just source analysis, I will directly use Visual Studio Code with C/C++ plug-in, also support symbol jump, find reference. In addition, I would like to remind you that the type hierarchy of C++ code in Swift stdlib is quite complex, and it would be quite difficult to read without the aid of IDE.

The body of the

Now we officially enter the source analysis phase, we first look at 1 Swift object (class instance) its memory layout is what.

HeapObject

We know that Objective-ES57en in runtime represents one object through objc_object, and these types define the structure of the object's head in memory. Similarly, there is a similar structure in Swift, which is HeapObject. Let's take a look at the definition of 1: 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};

As you can see, the first field of HeapObject is an HeapMetadata object, which does the same thing as isa_t and is used to describe the object type (equivalent to the result obtained by type(of:)), except that Swift does not use it in many cases, such as static method dispatch, etc.

Next is SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS, which is a macro definition. Expand it to: 


RefCounts<InlineRefCountBits> refCounts;

This is a very important thing, reference counting, weak references, unowned references are all related to it, and it is also a more complex structure in the Swift object (subsequent Swift objects in this article all guide to the type, i.e. instances of class).

Actually said complex also is not very complicated, we know Objective - C runtime has many union structure applications, such as isa_t pointer type and nonpointer types, they are all the same memory space occupied, the advantage is that it can more effectively use memory, especially the extensive use of the things, can greatly reduce the runtime overhead. A similar technique exists in JVM, as in mark word for object headers. Of course, this technique is also widely used in Swift ABI.

RefCounts type and Side Table

It says RefCounts type, so here's what it is.

Let's take a look at the definition of 1: 


template <typename RefCountBits>
class RefCounts {
 std::atomic<RefCountBits> refCounts;

 // ...

};

This is the memory layout of RefCounts, and I've omitted all method and type definitions here. You can think of RefCounts as a thread-safe wrapper. The template parameter RefCountBits specifies the actual internal type. There are two types in Swift ABI: 


typedef RefCounts<InlineRefCountBits> InlineRefCounts;
typedef RefCounts<SideTableRefCountBits> SideTableRefCounts;

The former is used in HeapObject and the latter in HeapObjectSideTableEntry (Side Table), both of which I will cover in 11.

In general, Swift objects do not use Side Table. Once an object is referenced by weak or unowned, the object is assigned an Side Table.

InlineRefCountBits

Definition: 


typedef RefCountBitsT<RefCountIsInline> InlineRefCountBits;

template <RefCountInlinedness refcountIsInline>
class RefCountBitsT {

 friend class RefCountBitsT<RefCountIsInline>;
 friend class RefCountBitsT<RefCountNotInline>;

 static const RefCountInlinedness Inlinedness = refcountIsInline;

 typedef typename RefCountBitsInt<refcountIsInline, sizeof(void*)>::Type
 BitsType;
 typedef typename RefCountBitsInt<refcountIsInline, sizeof(void*)>::SignedType
 SignedBitsType;
 typedef RefCountBitOffsets<sizeof(BitsType)>
 Offsets;

 BitsType bits;

 // ...

};

After template replacement, InlineRefCountBits is actually 1 uint64_t, and the associated 1 heap type is intended to make the code more readable (or less readable, hahaha) through template metaprogramming.

Let's simulate 1 object reference count +1:

Call the SIL interface swift::swift_retain: 


HeapObject *swift::swift_retain(HeapObject *object) {
 return _swift_retain(object);
}

static HeapObject *_swift_retain_(HeapObject *object) {
 SWIFT_RT_TRACK_INVOCATION(object, swift_retain);
 if (isValidPointerForNativeRetain(object))
 object->refCounts.increment(1);
 return object;
}

auto swift::_swift_retain = _swift_retain_;

Call the increment method of RefCounts: 


void increment(uint32_t inc = 1) {
 // 3.  Atomic readout  InlineRefCountBits  The object (i.e. 1 a  uint64_t ). 
 auto oldbits = refCounts.load(SWIFT_MEMORY_ORDER_CONSUME);
 RefCountBits newbits;
 do {
 newbits = oldbits;
 // 4.  call  InlineRefCountBits  the  incrementStrongExtraRefCount  methods 
 //  For this  uint64_t  for 1 Serial operations. 
 bool fast = newbits.incrementStrongExtraRefCount(inc);
 //  There is no  weak , unowned  reference 1 Usually won't enter. 
 if (SWIFT_UNLIKELY(!fast)) {
  if (oldbits.isImmortal())
  return;
  return incrementSlow(oldbits, inc);
 }
 // 5.  through  CAS  After the operation  uint64_t  Set it back. 
 } while (!refCounts.compare_exchange_weak(oldbits, newbits,
           std::memory_order_relaxed));
}

This is the end of the retain operation.

SideTableRefCountBits

The above is the case where weak and unowned references do not exist. Now let's see what happens if we add an weak reference.

Call the SIL interface swift::swift_weakAssign (skip the logic, it belongs to the logic of the referee, let's analyze the referee first) Call RefCounts < InlineRefCountBits > ::formWeakReference adds 1 weak quote: 

template <>
HeapObjectSideTableEntry* RefCounts<InlineRefCountBits>::formWeakReference()
{
 //  distribution 1 a  Side Table . 
 auto side = allocateSideTable(true);
 if (side)
 //  increase 1 A weak reference. 
 return side->incrementWeak();
 else
 return nullptr;
}

Focus on 1 Implementation of allocateSideTable: 


template <>
HeapObjectSideTableEntry* RefCounts<InlineRefCountBits>::allocateSideTable(bool failIfDeiniting)
{
 auto oldbits = refCounts.load(SWIFT_MEMORY_ORDER_CONSUME);

 //  existing  Side Table  Or is destructing on the direct return. 
 if (oldbits.hasSideTable()) {
 return oldbits.getSideTable();
 } 
 else if (failIfDeiniting && oldbits.getIsDeiniting()) {
 return nullptr;
 }

 //  distribution  Side Table  Object. 
 HeapObjectSideTableEntry *side = new HeapObjectSideTableEntry(getHeapObject());

 auto newbits = InlineRefCountBits(side);

 do {
 if (oldbits.hasSideTable()) {
  //  At this point another thread may have been created  Side Table , deletes the thread allocation, and returns. 
  auto result = oldbits.getSideTable();
  delete side;
  return result;
 }
 else if (failIfDeiniting && oldbits.getIsDeiniting()) {
  return nullptr;
 }

 //  With the current  InlineRefCountBits  Initialize the  Side Table . 
 side->initRefCounts(oldbits);
 //  for  CAS . 
 } while (! refCounts.compare_exchange_weak(oldbits, newbits,
            std::memory_order_release,
            std::memory_order_relaxed));
 return side;
}

Remember that RefCounts in HeapObject is actually an wrapper of InlineRefCountBits? After constructing Side Table above, InlineRefCountBits in the object is not the original reference count, but a pointer to Side Table, but since they are actually uint64_t, a method is required to distinguish them. To distinguish the difference, we can look at the constructor of InlineRefCountBits: 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};
0

In fact, the most common method is to replace the useless bit of pointer address with the identifier bit.

By the way, take a look at the structure of Side Table: 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};
1

What happens if I increase the reference count? Look at the RefCounts::increment method: 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};
2

At this point, we need to derive SideTableRefCountBits, which is similar to the previous InlineRefCountBits, but with an additional field, see the definition of 1: 


class SideTableRefCountBits : public RefCountBitsT<RefCountNotInline>
{
 uint32_t weakBits;

 // ...

};

Under the summary 1

I don't know if you are confused by the above content, but I spent some time when I started to analyze it.

There are two kinds of RefCounts. The first is inline, which is used in HeapObject. It is actually one uint64_t, which can be used as a reference count or as a pointer to Side Table.

Side Table is a structure of type 1 called HeapObjectSideTableEntry, which also has RefCounts members inside, which is the original uint64_t plus one uint32_t for storing weak references.

WeakReference

All of the above is the logic of the referenced object, but the logic of the referrer is a little simpler, mainly through the class WeakReference, which is a little simpler, we can do it under 1.

The weak variable in Swift is called swift::swift_weakAssign after passing through silgen and then sent to WeakReference::nativeAssign: 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};
4

Weak reference access is even simpler: 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};
5

Here you find one question, no, the referenced object is freed why can you access Side Table directly? In fact, the life cycle of Side Table in Swift ABI is separated from the object. When the strong reference count is 0, only HeapObject is released.

Side Table can only be released when all weak references are released or the related variables are set to nil. 


struct HeapObject {
 /// This is always a valid pointer to a metadata object.
 HeapMetadata const *metadata;

 SWIFT_HEAPOBJECT_NON_OBJC_MEMBERS;

 HeapObject() = default;

 // Initialize a HeapObject header as appropriate for a newly-allocated object.
 constexpr HeapObject(HeapMetadata const *newMetadata) 
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Initialized)
 { }

 // Initialize a HeapObject header for an immortal object
 constexpr HeapObject(HeapMetadata const *newMetadata,
      InlineRefCounts::Immortal_t immortal)
 : metadata(newMetadata)
 , refCounts(InlineRefCounts::Immortal)
 { }

};
6

So even if a weak reference is used, there is no guarantee that the associated memory will be freed entirely, since Side Table will exist as long as the weak variable is not explicitly set to nil. There is also an improvement in ABI: if you access a weak reference variable and find that the referenced object has been released, destroy your own weak reference to avoid repeating meaningless CAS operations later. Of course ABI does not do this optimization, we can also do it in Swift code. :)

conclusion

The above is a simple analysis of the implementation mode of Swift weak reference mechanism. It can be seen that the idea is very similar to Objective-ES303en runtime, both of which use Side Table matching objects to maintain reference count. The difference is that the ES307en-ES308en object has no Side Table pointer in the memory layout, but maintains the relationship between the object and Side Table through a global StripedMap, which is not as efficient as Swift. In addition, Objective-ES316en runtime releases the object with all succzero-ES320en variables, and Swift does not.

Overall, Swift is implemented in a slightly simpler way (although the code is more complex, the Swift team pursues higher abstractions). The first analysis of Swift ABI is for reference only. If there are any errors, you are welcome to correct them. Thank you!

Related articles: