background
In fact, this question is quite interesting. OutOfMemoryError is a common mistake. Apps, large and small, can never escape this Error. So, does OutOfMemroyError only happen when memory is allocated? Does it only happen when a new object is created? To clarify this problem, we need to understand the process of this Error.
OutOfMemoryError
Most of the OOM logs we often see in the stack are in the java layer. In fact, the OOM is actually set in the native method ThrowOutOfMemoryError
void Thread::ThrowOutOfMemoryError(const char* msg) {
LOG(WARNING) << "Throwing OutOfMemoryError "
<< '"' << msg << '"'
<< " (VmSize " << GetProcessStatus("VmSize")
<< (tls32_.throwing_OutOfMemoryError ? ", recursive case)" : ")");
ScopedTrace trace("OutOfMemoryError");
jni调用设置ERROR
if (!tls32_.throwing_OutOfMemoryError) {
tls32_.throwing_OutOfMemoryError = true;
ThrowNewException("Ljava/lang/OutOfMemoryError;", msg);
tls32_.throwing_OutOfMemoryError = false;
} else {
Dump(LOG_STREAM(WARNING)); // The pre-allocated OOME has no stack, so help out and log one.
SetException(Runtime::Current()->GetPreAllocatedOutOfMemoryErrorWhenThrowingOOME());
}
}
Next, let's take a look at several common paths that throw OOM
MakeSingleDexFile
In ART, it supports the synthesis of a single Dex. In the ClassPreDefine stage, it will try to generate a single Dex for qualified Classes (such as non-data/private classes). Here we will not go into the details of the process. Let’s take a look. If at this time Failure to move the old data orig_location to the new final_data array will trigger OOM
static std::unique_ptr<const art::DexFile> MakeSingleDexFile(art::Thread* self,
const char* descriptor,
const std::string& orig_location,
jint final_len,
const unsigned char* final_dex_data)
REQUIRES_SHARED(art::Locks::mutator_lock_) {
// Make the mmap
std::string error_msg;
art::ArrayRef<const unsigned char> final_data(final_dex_data, final_len);
art::MemMap map = Redefiner::MoveDataToMemMap(orig_location, final_data, &error_msg);
if (!map.IsValid()) {
LOG(WARNING) << "Unable to allocate mmap for redefined dex file! Error was: " << error_msg;
self->ThrowOutOfMemoryError(StringPrintf(
"Unable to allocate dex file for transformation of %s", descriptor).c_str());
return nullptr;
}
unsafe create
Our java layer also has a very magical class, which can also manipulate pointers, and can also directly create class objects and manipulate the memory pointer data of objects. It is Unsafe, and gson uses a lot of examples of unsafe to try to create objects. , for example, the object to be created has no empty parameter constructor, and if malloc fails to allocate memory, OOM will also be generated
static jlong Unsafe_allocateMemory(JNIEnv* env, jobject, jlong bytes) {
ScopedFastNativeObjectAccess soa(env);
if (bytes == 0) {
return 0;
}
// bytes is nonnegative and fits into size_t
if (!ValidJniSizeArgument(bytes)) {
DCHECK(soa.Self()->IsExceptionPending());
return 0;
}
const size_t malloc_bytes = static_cast<size_t>(bytes);
void* mem = malloc(malloc_bytes);
if (mem == nullptr) {
soa.Self()->ThrowOutOfMemoryError("native alloc");
return 0;
}
return reinterpret_cast<uintptr_t>(mem);
}
Thread creation
In fact, when we create a Thread in the Java layer, we will go to the native Thread creation. Through this method CreateNativeThread, in fact, the traditional pthread_create is called to create a native Thread. If the creation fails (such as insufficient virtual memory/FD), will go to the code block, resulting in OOM
void Thread::CreateNativeThread(JNIEnv* env, jobject java_peer, size_t stack_size, bool is_daemon) {
....
if (pthread_create_result == 0) {
// pthread_create started the new thread. The child is now responsible for managing the
// JNIEnvExt we created.
// Note: we can't check for tmp_jni_env == nullptr, as that would require synchronization
// between the threads.
child_jni_env_ext.release(); // NOLINT pthreads API.
return;
}
}
// Either JNIEnvExt::Create or pthread_create(3) failed, so clean up.
{
MutexLock mu(self, *Locks::runtime_shutdown_lock_);
runtime->EndThreadBirth();
}
// Manually delete the global reference since Thread::Init will not have been run. Make sure
// nothing can observe both opeer and jpeer set at the same time.
child_thread->DeleteJPeer(env);
delete child_thread;
child_thread = nullptr;
如果没有return，证明失败了，爆出OOM
SetNativePeer(env, java_peer, nullptr);
{
std::string msg(child_jni_env_ext.get() == nullptr ?
StringPrintf("Could not allocate JNI Env: %s", error_msg.c_str()) :
StringPrintf("pthread_create (%s stack) failed: %s",
PrettySize(stack_size).c_str(), strerror(pthread_create_result)));
ScopedObjectAccess soa(env);
soa.Self()->ThrowOutOfMemoryError(msg.c_str());
}
}
heap memory allocation
我们平时采用new 等方法的时候，其实进入到ART虚拟机中，其实是走到Heap::AllocObjectWithAllocator 这个方法里面，当内存分配不足的时候，就会发起一次强有力的gc后再尝试进行内存分配，这个方法就是AllocateInternalWithGc
mirror::Object* Heap::AllocateInternalWithGc(Thread* self,
AllocatorType allocator,
bool instrumented,
size_t alloc_size,
size_t* bytes_allocated,
size_t* usable_size,
size_t* bytes_tl_bulk_allocated,
ObjPtr<mirror::Class>* klass)
流程如下：
void Heap::ThrowOutOfMemoryError(Thread* self, size_t byte_count, AllocatorType allocator_type) {
// If we're in a stack overflow, do not create a new exception. It would require running the
// constructor, which will of course still be in a stack overflow.
if (self->IsHandlingStackOverflow()) {
self->SetException(
Runtime::Current()->GetPreAllocatedOutOfMemoryErrorWhenHandlingStackOverflow());
return;
}
这里官方给了一个钩子
Runtime::Current()->OutOfMemoryErrorHook();
输出OOM的原因
std::ostringstream oss;
size_t total_bytes_free = GetFreeMemory();
oss << "Failed to allocate a " << byte_count << " byte allocation with " << total_bytes_free
<< " free bytes and " << PrettySize(GetFreeMemoryUntilOOME()) << " until OOM,"
<< " target footprint " << target_footprint_.load(std::memory_order_relaxed)
<< ", growth limit "
<< growth_limit_;
// If the allocation failed due to fragmentation, print out the largest continuous allocation.
if (total_bytes_free >= byte_count) {
space::AllocSpace* space = nullptr;
if (allocator_type == kAllocatorTypeNonMoving) {
space = non_moving_space_;
} else if (allocator_type == kAllocatorTypeRosAlloc ||
allocator_type == kAllocatorTypeDlMalloc) {
space = main_space_;
} else if (allocator_type == kAllocatorTypeBumpPointer ||
allocator_type == kAllocatorTypeTLAB) {
space = bump_pointer_space_;
} else if (allocator_type == kAllocatorTypeRegion ||
allocator_type == kAllocatorTypeRegionTLAB) {
space = region_space_;
}
// There is no fragmentation info to log for large-object space.
if (allocator_type != kAllocatorTypeLOS) {
CHECK(space != nullptr) << "allocator_type:" << allocator_type
<< " byte_count:" << byte_count
<< " total_bytes_free:" << total_bytes_free;
// LogFragmentationAllocFailure returns true if byte_count is greater than
// the largest free contiguous chunk in the space. Return value false
// means that we are throwing OOME because the amount of free heap after
// GC is less than kMinFreeHeapAfterGcForAlloc in proportion of the heap-size.
// Log an appropriate message in that case.
if (!space->LogFragmentationAllocFailure(oss, byte_count)) {
oss << "; giving up on allocation because <"
<< kMinFreeHeapAfterGcForAlloc * 100
<< "% of heap free after GC.";
}
}
}
self->ThrowOutOfMemoryError(oss.str().c_str());
}
这个就是我们常见的，也是主要OOM产生的流程
JNI层
这里还有很多，比如JNI层通过Env调用NewString等分配内存的时候，会进入条件检测，比如分配的String长度超过最大时产生Error，即使说内存空间依旧可以分配，但是超过了虚拟机能处理的最大限制，也会产生OOM
if (UNLIKELY(utf16_length > static_cast<uint32_t>(std::numeric_limits<int32_t>::max()))) {
// Converting the utf16_length to int32_t would overflow. Explicitly throw an OOME.
std::string error =
android::base::StringPrintf("NewStringUTF input has 2^31 or more characters: %zu",
utf16_length);
ScopedObjectAccess soa(env);
soa.Self()->ThrowOutOfMemoryError(error.c_str());
return nullptr;
}
OOM 路径总结
通过本文，我们看到了OOM发生时，可能存在的几个主要路径，其他引起OOM的路径，也是在这几个基础路径之上产生的，希望大家以后可以带着源码学习，能够帮助我们了解ART更深层的秘密。