Win32 ProgrammingLesson 18: More Memory Mapped Files and the HEAP(Finally, cool stuff!)

Where are we? We’ve gotten pretty familiar with the idea of

memory-mapped files but there are some important concepts we haven’t looked at

Finish up today, and then look at the heap

Sharing Data There are lots of different ways to share data

between processes But the “lowest level” way is really in

memory, via a memory-mapped file Two or more processes map the same base

address – hence they are sharing the same physical memory

Avoiding the “real” File system Very inconvenient if every memory-mapped

file actually had to exist on disk Imagine that you simply wanted to use the

mechanism to pass data, not keep it Can call CreateFileMapping with

INVALID_HANDLE_VALUE as the hFile parameter, and the memory-mapped file is backed by the page file

30 Second Quiz What’s wrong with this code? HANDLE hFile = CreateFile(...);

HANDLE hMap = CreateFileMapping(hFile, ...); if (hMap == NULL)    return(GetLastError());

Answer… You’ll get back

INVALID_HANDLE_VALUE from the first call

Which means…

MMFExample Simple program When it maps the view of the file, it transfers

data between the two programs Nice method of sharing between two

processes!

Sparsely Committing… Remember we talked about how to commit

memory for files? Same discussion for Memory-mapped files –

that is, we don’t need to commit all our memory at once

Consider CELLDATA CellData[200][256]; If sizeof(CELLDATA) is 128 that’s about

6MB. Better to share as a sparsely-committed file

mapping object If we’re sharing via the paging file, can use

SEC_RESERVE or SEC_COMMIT

SEC_RESERVE When you pass in SEC_RESERVE you don’t

actually commit the space Just returns a HANDLE to the file mapping

object Any attempts to access the memory cause a

memory violation

VirtualAlloc (again) Solution: Call VirtualAlloc to allocate the

memory as we use it!

The HEAP Heap is a fantastic tool for allocating small

blocks of memory Perfect for linked lists and trees Advantage: can ignore allocation granularity Disadvantage: slow, with no direct control of

physical allocation Better yet, internals not entirely documented

Default Each process gets a default heap of 1MB Can specify at link time (/HEAP:) Used by many Windows/C RTL functions Access to the HEAP is serialized (why, and

what does this mean?) Can obtain a handle via:

HANDLE GetProcessHeap();

More than 1 Heap is a… Five reasons you might want to do this:

Component protection More efficient memory management Local access Avoiding thread sync overhead Quick free

1: Component Protection Imagine you have two structures: a linked list

and a binary tree If you share one heap, and one has a bug, the

problem may show up in the other structure If we have different heaps, problems tend to

be localized (unless you *really* mess up!)

2: Memory Management Heaps work best when all the objects in them

are the same size Imagine mixing two different sizes; when you

free object 1 object 2 may not be a perfect fit Better to allocate all objects of the same size

in the same place

3: Local Access Swapping to disk is really expensive Best to keep things you use together close

together

4: Avoiding thread-sync issues Heaps are serialized by default CPU overhead involved in keeping heap

access thread safe If you tell the system a heap is single-

threaded, you can lose this overhead DANGER WILL ROBINSON: YOU ARE NOW

RESPONSIBLE FOR THREAD SAFETY!!!

5: Quick free Instead of freeing things up block by block

you can choose to free the entire heap in one go

That’s *really* quick

So… how? HANDLE HeapCreate(

DWORD fdwOptions, SIZE_T dwInitialSize, SIZE_T dwMaximumSize);

Options: 0, HEAP_NO_SERIALIZE, HEAP_GENERATE_EXCEPTIONS

Serialize turns off checking for Alloc and Free Can manage this yourself via Critical Sections if you

want to

HEAP_GENERATE_EXCEPTIONS Raise an exception whenever an allocation

request fails Basically, it’s just about whether you want to

catch exceptions or check return values – depends on the application

Oh… if dwMaximumSize is 0 the size is unlimited…

Allocating Memory from the Heap PVOID HeapAlloc(

HANDLE hHeap, DWORD fdwFlags, SIZE_T dwBytes);

Flags: HEAP_ZERO_MEMORY, HEAP_GENERATE_EXCEPTIONS, HEAP_NO_SERIALIZE

Exceptions: STATUS_NO_MEMORY, STATUS_ACCESS_VIOLATION

Changing the size… PVOID HeapReAlloc(

HANDLE hHeap, DWORD fdwFlags, PVOID pvMem, SIZE_T dwBytes);

New flag: HEAP_REALLOC_IN_PLACE_ONLY

Means that the location won’t change

Obtaining the Size SIZE_T HeapSize(

HANDLE hHeap, DWORD fdwFlags, LPCVOID pvMem);

Flags: 0 or HEAP_NO_SERIALIZE

Freeing a block BOOL HeapFree(

HANDLE hHeap, DWORD fdwFlags, PVOID pvMem);

Flags? You tell me…

Destroying a Heap BOOL HeapDestroy(HANDLE hHeap); TRUE on success You can’t destroy the default heap!

Heaps with C++ Under C you would use malloc In C++ can use new/delete

CSomeClass *pSomeClass = new CSomeClass; delete pSomeClass; Now the clever bit: overload new/delete…

Prototype class CSomeClass {

private:    static HANDLE s_hHeap;    static UINT s_uNumAllocsInHeap;

// Other private data and member functions public:    

void* operator new (size_t size);    void operator delete (void* p);    

// Other public data and member functions };

Code…HANDLE CSomeClass::s_hHeap = NULL;UINT CSomeClass::s_uNumAllocsInHeap = 0; void* CSomeClass::operator new (size_t size) {    if (s_hHeap == NULL) {       // Heap does not exist; create it.       s_hHeap = HeapCreate(HEAP_NO_SERIALIZE, 0, 0);       if (s_hHeap == NULL)          return(NULL);    }    // The heap exists for CSomeClass objects.    void* p = HeapAlloc(s_hHeap, 0, size);    if (p != NULL) {       // Memory was allocated successfully; increment // the count of CSomeClass objects in the heap.       s_uNumAllocsInHeap++;    }    // Return the address of the allocated CSomeClass object.    return(p);}

And delete…void CSomeClass::operator delete (void* p) {

   if (HeapFree(s_hHeap, 0, p)) {       // Object was deleted successfully.       s_uNumAllocsInHeap--;    }    if (s_uNumAllocsInHeap == 0) {       // If there are no more objects in the heap,       // destroy the heap.       if (HeapDestroy(s_hHeap)) {          // Set the heap handle to NULL

// so that the new operator          // will know to create a new heap if a 

// new CSomeClass          // object is created.          s_hHeap = NULL;       }    }

}

Misc Functions… DWORD GetProcessHeaps – returns handles to all

heaps in the process BOOL HeapValidate – check that a heap is a-okay… UINT HeapCompact – coalesce free blocks HeapLock and HeapUnlock – used for thread sync HeapWalk – for debugging; lets you enumerate

sections/blocks in the heap

Next Next, it gets difficult

DLLs