为什么在 Win32 控制台应用程序启动时会出现三个

LdrpInitializeProcess() calls NTSTATUS LdrpEnableParallelLoading (ULONG LoaderThreads) - as LdrpEnableParallelLoading(ProcessParameters->LoaderThreads):

NTSTATUS LdrpEnableParallelLoading (ULONG LoaderThreads)
{
    LdrpDetectDetour();

    if (LoaderThreads)
    {
        LoaderThreads = min(LoaderThreads, 16);// not more than 16 threads allowed
        if (LoaderThreads <= 1) return STATUS_SUCCESS;
    }
    else
    {
        if (RtlGetSuiteMask() & 0x10000) return STATUS_SUCCESS; 
        LoaderThreads = 4;// default for 4 threads
    }

    if (LdrpDetourExist) return STATUS_SUCCESS;

    NTSTATUS status = TpAllocPool(&LdrpThreadPool, 1);//CreateThreadpool

    if (0 <= status)
    {
        TpSetPoolWorkerThreadIdleTimeout(LdrpThreadPool, -300000000);// 30 second idle timeout
        TpSetPoolMaxThreads(LdrpThreadPool, LoaderThreads - 1);//SetThreadpoolThreadMaximum 
        TP_CALLBACK_ENVIRON CallbackEnviron = { };
        CallbackEnviron->CallbackPriority = TP_CALLBACK_PRIORITY_NORMAL;
        CallbackEnviron->Size = sizeof(TP_CALLBACK_ENVIRON);
        CallbackEnviron->Pool = LdrpThreadPool;
        CallbackEnviron->Version = 3;

        status = TpAllocWork(&LdrpMapAndSnapWork, LdrpWorkCallback, 0, &CallbackEnviron);//CreateThreadpoolWork
    }

    return status;
}

创建了一个特殊的加载器线程池 - LdrpThreadPool，具有 LoaderThreads - 1 个最大线程.空闲超时设置为30秒(之后线程退出)并分配PTP_WORK LdrpMapAndSnapWork，然后在void LdrpQueueWork(LDRP_LOAD_CONTEXT* LoadContext)中使用.

A special loader thread pool is created - LdrpThreadPool, with LoaderThreads - 1 max threads. Idle timeout is set to 30 seconds (after which the thread exits) and allocated PTP_WORK LdrpMapAndSnapWork, which is then used in void LdrpQueueWork(LDRP_LOAD_CONTEXT* LoadContext).

并行加载器使用的全局变量:

Global variables used by the parallel loader:

HANDLE LdrpWorkCompleteEvent, LdrpLoadCompleteEvent;
CRITICAL_SECTION LdrpWorkQueueLock;
LIST_ENTRY LdrpWorkQueue = { &LdrpWorkQueue, &LdrpWorkQueue };


ULONG LdrpWorkInProgress;
BOOLEAN LdrpDetourExist;
PTP_POOL LdrpThreadPool;

PTP_WORK LdrpMapAndSnapWork;

enum DRAIN_TASK {
    WaitLoadComplete, WaitWorkComplete
};

struct LDRP_LOAD_CONTEXT
{
    UNICODE_STRING BaseDllName;
    PVOID somestruct;
    ULONG Flags;//some unknown flags
    NTSTATUS* pstatus; //final status of load
    _LDR_DATA_TABLE_ENTRY* ParentEntry; // of 'parent' loading dll
    _LDR_DATA_TABLE_ENTRY* Entry; // this == Entry->LoadContext
    LIST_ENTRY WorkQueueListEntry;
    _LDR_DATA_TABLE_ENTRY* ReplacedEntry;
    _LDR_DATA_TABLE_ENTRY** pvImports;// in same ordef as in IMAGE_IMPORT_DESCRIPTOR piid
    ULONG ImportDllCount;// count of pvImports
    LONG TaskCount;
    PVOID pvIAT;
    ULONG SizeOfIAT;
    ULONG CurrentDll; // 0 <= CurrentDll < ImportDllCount
    PIMAGE_IMPORT_DESCRIPTOR piid;
    ULONG OriginalIATProtect;
    PVOID GuardCFCheckFunctionPointer;
    PVOID* pGuardCFCheckFunctionPointer;
};

不幸的是 LDRP_LOAD_CONTEXT 未包含在已发布的 .pdb 文件中，因此我的定义仅包含部分名称.

Unfortunately LDRP_LOAD_CONTEXT is not contained in published .pdb files, so my definitions include only partial names.

struct {
    ULONG MaxWorkInProgress;//4 - values from explorer.exe at some moment
    ULONG InLoaderWorker;//7a (this mean LdrpSnapModule called from worker thread)
    ULONG InLoadOwner;//87 (LdrpSnapModule called direct, in same thread as `LdrpMapAndSnapDependency`)
} LdrpStatistics;

// for statistics
void LdrpUpdateStatistics()
{
  LdrpStatistics.MaxWorkInProgress = max(LdrpStatistics.MaxWorkInProgress, LdrpWorkInProgress);
  NtCurrentTeb()->LoaderWorker ? LdrpStatistics.InLoaderWorker++ : LdrpStatistics.InLoadOwner++
}

在 TEB.CrossTebFlags - 现在存在 2 个新标志:

In TEB.CrossTebFlags - now exist 2 new flags:

USHORT LoadOwner : 01; // 0x1000;
USHORT LoaderWorker : 01; // 0x2000;

最后 2 位是空闲的 (USHORT SpareSameTebBits : 02;//0xc000)

Last 2 bits is spare (USHORT SpareSameTebBits : 02; // 0xc000)

LdrpMapAndSnapDependency(LDRP_LOAD_CONTEXT* LoadContext) 包括以下代码:

LDR_DATA_TABLE_ENTRY* Entry = LoadContext->CurEntry;
if (LoadContext->pvIAT)
{
    Entry->DdagNode->State = LdrModulesSnapping;
    if (LoadContext->PrevEntry)// if recursive call
    {
        LdrpQueueWork(LoadContext); // !!!
    }
    else
    {
        status = LdrpSnapModule(LoadContext);
    }
}
else
{
    Entry->DdagNode->State = LdrModulesSnapped;
}

所以，如果 LoadContext->PrevEntry(假设我们加载 user32.dll.在第一次调用 LdrpMapAndSnapDependency() 时，LoadContext->PrevEntry 将始终为 0(当 CurEntry 指向 user32.dll 时)，但是当我们递归调用 LdrpMapAndSnapDependency() 依赖 gdi32.dll - PrevEntry 将用于 user32.dll 和 CurEntry 用于 gdi32.dll)，我们不直接调用LdrpSnapModule(LoadContext);而是LdrpQueueWork(LoadContext);.

So, if LoadContext->PrevEntry (say we load user32.dll. In the first call to LdrpMapAndSnapDependency(), LoadContext->PrevEntry will be always 0 (when CurEntry points to user32.dll), but when we recursively call LdrpMapAndSnapDependency() for it dependency gdi32.dll - PrevEntry will be for user32.dll and CurEntry for gdi32.dll), we do not direct call LdrpSnapModule(LoadContext); but LdrpQueueWork(LoadContext);.

LdrpQueueWork() 很简单:

void LdrpQueueWork(LDRP_LOAD_CONTEXT* LoadContext)
{
    if (0 <= ctx->pstatus)
    {
        EnterCriticalSection(&LdrpWorkQueueLock);

        InsertHeadList(&LdrpWorkQueue, &LoadContext->WorkQueueListEntry);

        LeaveCriticalSection(&LdrpWorkQueueLock);

        if (LdrpMapAndSnapWork && !RtlGetCurrentPeb()->Ldr->ShutdownInProgress)
        {
            SubmitThreadpoolWork(LdrpMapAndSnapWork);//TpPostWork
        }
    }
}

我们将 LoadContext 插入到 LdrpWorkQueue 中，如果Parallel loader"已启动 (LdrpMapAndSnapWork != 0) 而不是 ShutdownInProgress - 我们将工作提交到加载器池.但是即使池没有初始化(比如因为 Detours 存在) - 也不会出现错误 - 我们在 LdrpDrainWorkQueue() 中处理这个任务.

We insert LoadContext to LdrpWorkQueue and if "Parallel loader" is started (LdrpMapAndSnapWork != 0) and not ShutdownInProgress - we submit work to loader pool. But even if the pool is not initialized (say because Detours exist) - there will be no error - we process this task in LdrpDrainWorkQueue().

在工作线程回调中执行:

In a worker thread callback, this is executed:

void LdrpWorkCallback()
{
    if (LdrpDetourExist) return;

    EnterCriticalSection(&LdrpWorkQueueLock);

    PLIST_ENTRY Entry = RemoveEntryList(&LdrpWorkQueue);

    if (Entry != &LdrpWorkQueue)
    {
        ++LdrpWorkInProgress;
        LdrpUpdateStatistics()
    }

    LeaveCriticalSection(&LdrpWorkQueueLock);

    if (Entry != &LdrpWorkQueue)
    {
        LdrpProcessWork(CONTAINING_RECORD(Entry, LDRP_LOAD_CONTEXT, WorkQueueListEntry), FALSE);
    }
}