mirrors/reactos
0
0
Fork 0
mirror of https://github.com/reactos/reactos.git synced 2025-01-20 18:44:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

[NEWCC]

Browse Source 
A reintegration checkpoint for the NewCC branch, brought to you by Team NewCC.

Differences with current ReactOS trunk:

* A new memory area type, MEMORY_AREA_CACHE, is added, which represents a mapped region of a file. In NEWCC mode, user sections are MEMORY_AREA_CACHE type as well, and obey new semantics. In non-NEWCC mode, they aren't used.
* A way of claiming a page entry for a specific thread's work is added. Placing the special SWAPENTRY value MM_WAIT_ENTRY in a page table, or in a section page table should indicate that memory management code is intended to wait for another thread to make some status change before checking the state of the page entry again. In code that uses this convention, a return value of STATUS_SUCCESS + 1 is used to indicate that the caller should use the MiWaitForPageEvent macro to wait until somebody has change the state of a wait entry before checking again. This is a lighter weight mechanism than PAGEOPs.
* A way of asking the caller to perform some blocking operation without locks held is provided. This replaces some spaghettified code in which locks are repeatedly taken and broken by code that performs various blocking operations. Using this mechanism, it is possible to do a small amount of non-blocking work, fill in a request, then return STATUS_MORE_PROCESSING_REQUIRED to request that locks be dropped and the blocking operation be carried out. A MM_REQUIRED_RESOURCES structure is provided to consumers of this contract to use to accumulate state across many blocking operations. Several functions wrapping blocking operations are provided in ntoskrnl/cache/reqtools.c.
* Image section pages are no longer direct mapped. This is done to simplify consolidation of ownership of pages under the data section system. At a later time, it may be possible to make data pages directly available to image sections for the same file. This is likely the only direct performance impact this code makes on non-NEWCC mode.

RMAPs:

* A new type of RMAP entry is introduced, distinguished by RMAP_IS_SEGMENT(Address) of the rmap entry. This kind of entry contains a pointer to a section page table node in the Process pointer, which in turn links back to the MM_SECTION_SEGMENT it belongs to. Therefore, a page belonging only to a segment (that is, a segment page that isn't mapped) can exist and be evicted using the normal page eviction mechanism in balance.c. Each of the rmap function has been modified to deal with segment rmaps.
* The low 8 bits of the Address field in a segment rmap denote the entry number in the generic table node pointed to by Process that points to the page the rmap belongs to. By combining them, you can determine the file offset the page belongs to.
* In NEWCC mode, MmSharePageEntry/UnsharePageEntry are not used, and instead the page reference count is used to keep track of the number of mappings of a page, allowing the last reference expiring to allow the page to be recycled without much intervention. These are still used in non-NEWCC mode. One change has been made, the count fields have been narrowed by 1 bit to make room for a dirty bit in SSE entries, needed when a page is present but unmapped.

Section page tables:

* The section page tables are now implemented using RtlGenericTables. This enables a fairly compact representation of section page tables without having the existence of a section object imply 4k of fake PDEs. In addition, each node in the generic table has a wide file offset that is a multiple of 256 pages, or 1 megabyte total. Besides needing wide file offsets, the only other visible change caused by the switch to generic tables for section page tables is the need to lock the section segment before interacting with the section page table.

Eviction:

* Page eviction in cache sections is accomplished by MmpPageOutPhysicalAddress. In the case of a shared page, it tries to remove all mappings of the indicated page. If this process fails at any point, the page will simply be drawn back into the target address spaces. After succeeding at this, if TRUE has been accumulated into the page's dirty bit in the section page table, it is written back, and then permanently removed.

NewCC mode:

* NEWCC mode is introduced, which rewrites the file cache to a set of cache stripes actively mapped, along with unmapped section data.
* NewCC is more authentic in its interpretation of the external interface to the windows cache than the current cache manager, implementing each of the cache manager functions according to the documented interface with no preconceived ideas about how anything should be implemented internally. Cache stripes are implemented on top of section objects, using the same memory manager paths, and therefore economizing code and complexity. This replaces a rather complicated system in which pages can be owned by the cache manager and the memory manager simultaneously and they must cooperate in a fairly sophisticated way to manage them. Since they're quite interdependent in the current code, modifying either is very difficult. In NEWCC, they have a clear division of labor and thus can be worked on independently.
* Several third party filesystems that use the kernel Cc interface work properly using NEWCC, including matt wu's ext3 driver.
* In contrast with code that tries to make CcInitializeCacheMap and CcUninitializeCacheMap into a pair that supports reference counting, NEWCC lazily initializes the shared and private cache maps as needed and uses the presence of a PrivateCacheMap on at least one file pointing to the SharedCacheMap as an indication that the FILE_OBJECT reference in the SharedCacheMap should still be held. When the last PrivateCacheMap is discarded, that's the appropriate time to tear down caching for a specific file, as the SharedCacheMap data is allowed to be saved and reused. We honor this by making the SharedCacheMap into a depot for keeping track of the PrivateCacheMap objects associated with views of a file.

svn path=/trunk/; revision=55833
This commit is contained in:
Amine Khaldi 2012-02-23 12:03:06 +00:00
parent 8cf1d52f96
commit e14f67f95d
27 changed files with 1087 additions and 954 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
reactos/ntoskrnl/CMakeLists.txt
View File

@ -18,7 +18,9 @@ add_definitions(
set_rc_compiler()
set(NEWCC FALSE)
if(NOT DEFINED NEWCC)
set(NEWCC FALSE)
endif(NOT DEFINED NEWCC)
if(NEWCC)
add_definitions(-DNEWCC)
@ -29,12 +31,7 @@ if(NEWCC)
cache/lazyrite.c
cache/logsup.c
cache/mdlsup.c
cache/pinsup.c
cache/section/data.c
cache/section/fault.c
cache/section/reqtools.c
cache/section/sptab.c
cache/section/swapout.c)
cache/pinsup.c)
else()
list(APPEND SOURCE
cc/cacheman.c
@ -47,6 +44,11 @@ endif()
list(APPEND SOURCE
cache/section/io.c
cache/section/data.c
cache/section/fault.c
cache/section/reqtools.c
cache/section/sptab.c
cache/section/swapout.c
config/cmalloc.c
config/cmapi.c
config/cmboot.c

16
reactos/ntoskrnl/cache/cachesub.c vendored
View File

@ -12,7 +12,7 @@
#include <ntoskrnl.h>
#include "newcc.h"
#include "section/newmm.h"
#define NDEBUG
//#define NDEBUG
#include <debug.h>
/* STRUCTURES *****************************************************************/
@ -143,9 +143,11 @@ _CcpFlushCache(IN PNOCC_CACHE_MAP Map,
PNOCC_BCB Bcb = NULL;
LARGE_INTEGER LowerBound, UpperBound;
PLIST_ENTRY ListEntry;
IO_STATUS_BLOCK IOSB = {0};
IO_STATUS_BLOCK IOSB;
DPRINT1("CcFlushCache (while file) (%s:%d)\n", File, Line);
RtlZeroMemory(&IOSB, sizeof(IO_STATUS_BLOCK));
DPRINT("CcFlushCache (while file) (%s:%d)\n", File, Line);
if (FileOffset && Length)
{
@ -288,7 +290,8 @@ VOID
NTAPI
CcShutdownSystem()
{
ULONG i;
ULONG i, Result;
NTSTATUS Status;
DPRINT1("CC: Shutdown\n");
@ -308,7 +311,10 @@ CcShutdownSystem()
}
}
DPRINT1("Done\n");
// Evict all section pages
Status = MiRosTrimCache(~0, 0, &Result);
DPRINT1("Done (Evicted %d, Status %x)\n", Result, Status);
}

6
reactos/ntoskrnl/cache/copysup.c vendored
View File

@ -11,7 +11,7 @@
#include <ntoskrnl.h>
#include "newcc.h"
#include "section/newmm.h"
#define NDEBUG
//#define NDEBUG
#include <debug.h>
/* GLOBALS ********************************************************************/
@ -80,7 +80,7 @@ CcCopyRead(IN PFILE_OBJECT FileObject,
return FALSE;
}
DPRINT1("Copying %d bytes at %08x%08x\n", ReadLen, CacheOffset.HighPart, CacheOffset.LowPart);
DPRINT("Copying %d bytes at %08x%08x\n", ReadLen, CacheOffset.HighPart, CacheOffset.LowPart);
RtlCopyMemory
(BufferTarget,
ReadBuffer,
@ -159,7 +159,7 @@ CcCopyWrite(IN PFILE_OBJECT FileObject,
DPRINT1("CcPreparePinWrite Failed?\n");
if (Wait) RtlRaiseStatus(STATUS_NOT_MAPPED_DATA); else return FALSE;
}
DPRINT1("Copying actual memory to BCB#%x (@%x) (from buffer at %x)\n", Bcb - CcCacheSections, WriteBuf, Bcb->BaseAddress);
DPRINT("Copying actual memory to BCB#%x (@%x) (from buffer at %x)\n", Bcb - CcCacheSections, WriteBuf, Bcb->BaseAddress);
//MiZeroFillSection(WriteBuf, &CurrentOffset, WriteLen);
RtlCopyMemory(WriteBuf, ((PCHAR)Buffer) + Count, WriteLen);

56
reactos/ntoskrnl/cache/fssup.c vendored
View File

@ -12,7 +12,7 @@
#include <ntoskrnl.h>
#include "newcc.h"
#include "section/newmm.h"
#define NDEBUG
//#define NDEBUG
#include <debug.h>
/* GLOBALS ********************************************************************/
@ -38,44 +38,6 @@ LIST_ENTRY CcpAllSharedCacheMaps;
/* FUNCTIONS ******************************************************************/
// Interact with legacy balance manager for now
// This can fall away when our section implementation supports
// demand paging properly
NTSTATUS
CcRosTrimCache(ULONG Target, ULONG Priority, PULONG NrFreed)
{
ULONG i, Freed, BcbHead;
*NrFreed = 0;
for (i = 0; i < CACHE_NUM_SECTIONS; i++) {
BcbHead = (i+CcCacheClockHand) % CACHE_NUM_SECTIONS;
// Reference a cache stripe so it won't go away
CcpLock();
if (CcCacheSections[BcbHead].BaseAddress) {
CcpReferenceCache(BcbHead);
CcpUnlock();
} else {
CcpUnlock();
continue;
}
// Defer to MM to try recovering pages from it
Freed = MiCacheEvictPages
(CcCacheSections[BcbHead].BaseAddress, Target);
Target -= Freed;
*NrFreed += Freed;
CcpLock();
CcpUnpinData(&CcCacheSections[BcbHead], TRUE);
CcpUnlock();
}
return STATUS_SUCCESS;
}
BOOLEAN
NTAPI
CcInitializeCacheManager(VOID)
@ -99,14 +61,6 @@ CcInitializeCacheManager(VOID)
CcCacheBitmap->SizeOfBitMap = ROUND_UP(CACHE_NUM_SECTIONS, 32);
DPRINT("Cache has %d entries\n", CcCacheBitmap->SizeOfBitMap);
ExInitializeFastMutex(&CcMutex);
ExInitializeFastMutex(&GlobalPageOperation);
// MM stub
KeInitializeEvent(&MmWaitPageEvent, SynchronizationEvent, FALSE);
// Until we're fully demand paged, we can do things the old way through
// the balance manager
MmInitializeMemoryConsumer(MC_CACHE, CcRosTrimCache);
return TRUE;
}
@ -268,9 +222,9 @@ CcUninitializeCacheMap(IN PFILE_OBJECT FileObject,
RemoveEntryList(&PrivateCacheMap->ListEntry);
if (IsListEmpty(&PrivateCacheMap->Map->PrivateCacheMaps))
{
while (!IsListEmpty(&Map->AssociatedBcb))
while (!IsListEmpty(&PrivateCacheMap->Map->AssociatedBcb))
{
PNOCC_BCB Bcb = CONTAINING_RECORD(Map->AssociatedBcb.Flink, NOCC_BCB, ThisFileList);
PNOCC_BCB Bcb = CONTAINING_RECORD(PrivateCacheMap->Map->AssociatedBcb.Flink, NOCC_BCB, ThisFileList);
DPRINT("Evicting cache stripe #%x\n", Bcb - CcCacheSections);
Bcb->RefCount = 1;
CcpDereferenceCache(Bcb - CcCacheSections, TRUE);
@ -383,7 +337,7 @@ CcZeroData(IN PFILE_OBJECT FileObject,
{
ToWrite = MIN(UpperBound.QuadPart - LowerBound.QuadPart, (PAGE_SIZE - LowerBound.QuadPart) & (PAGE_SIZE - 1));
DPRINT("Zero last half %08x%08x %x\n", Target.u.HighPart, Target.u.LowPart, ToWrite);
Status = MiSimpleRead(FileObject, &Target, ZeroBuf, PAGE_SIZE, &IOSB);
Status = MiSimpleRead(FileObject, &Target, ZeroBuf, PAGE_SIZE, TRUE, &IOSB);
if (!NT_SUCCESS(Status))
{
ExFreePool(ZeroBuf);
@ -419,7 +373,7 @@ CcZeroData(IN PFILE_OBJECT FileObject,
{
ToWrite = UpperBound.QuadPart - Target.QuadPart;
DPRINT("Zero first half %08x%08x %x\n", Target.u.HighPart, Target.u.LowPart, ToWrite);
Status = MiSimpleRead(FileObject, &Target, ZeroBuf, PAGE_SIZE, &IOSB);
Status = MiSimpleRead(FileObject, &Target, ZeroBuf, PAGE_SIZE, TRUE, &IOSB);
if (!NT_SUCCESS(Status))
{
ExFreePool(ZeroBuf);

2
reactos/ntoskrnl/cache/lazyrite.c vendored
View File

@ -10,7 +10,7 @@
#include <ntoskrnl.h>
#include "newcc.h"
#define NDEBUG
//#define NDEBUG
#include <debug.h>
/* GLOBALS ********************************************************************/

2
reactos/ntoskrnl/cache/logsup.c vendored
View File

@ -10,7 +10,7 @@
#include <ntoskrnl.h>
#include "newcc.h"
#define NDEBUG
//#define NDEBUG
#include <debug.h>
/* GLOBALS ********************************************************************/

2
reactos/ntoskrnl/cache/mdlsup.c vendored
View File

@ -10,7 +10,7 @@
#include <ntoskrnl.h>
#include "newcc.h"
#define NDEBUG
//#define NDEBUG
#include <debug.h>
/* GLOBALS ********************************************************************/

4
reactos/ntoskrnl/cache/newcc.h vendored
View File

@ -1,14 +1,12 @@
#pragma once
struct _MM_CACHE_SECTION_SEGMENT;
typedef struct _NOCC_BCB
{
/* Public part */
PUBLIC_BCB Bcb;
struct _NOCC_CACHE_MAP *Map;
struct _MM_CACHE_SECTION_SEGMENT *SectionObject;
PROS_SECTION_OBJECT SectionObject;
LARGE_INTEGER FileOffset;
ULONG Length;
PVOID BaseAddress;

46
reactos/ntoskrnl/cache/pinsup.c vendored
View File

@ -59,7 +59,7 @@ NTSTATUS CcpAllocateSection
(PFILE_OBJECT FileObject,
ULONG Length,
ULONG Protect,
PMM_CACHE_SECTION_SEGMENT *Result)
PROS_SECTION_OBJECT *Result)
{
NTSTATUS Status;
LARGE_INTEGER MaxSize;
@ -68,13 +68,14 @@ NTSTATUS CcpAllocateSection
DPRINT("Making Section for File %x\n", FileObject);
DPRINT("File name %wZ\n", &FileObject->FileName);
Status = MmCreateCacheSection
(Result,
Status = MmCreateSection
((PVOID*)Result,
STANDARD_RIGHTS_REQUIRED,
NULL,
&MaxSize,
Protect,
SEC_RESERVE | SEC_CACHE,
NULL,
FileObject);
return Status;
@ -86,7 +87,7 @@ typedef struct _WORK_QUEUE_WITH_CONTEXT
PVOID ToUnmap;
LARGE_INTEGER FileOffset;
LARGE_INTEGER MapSize;
PMM_CACHE_SECTION_SEGMENT ToDeref;
PROS_SECTION_OBJECT ToDeref;
PACQUIRE_FOR_LAZY_WRITE AcquireForLazyWrite;
PRELEASE_FROM_LAZY_WRITE ReleaseFromLazyWrite;
PVOID LazyContext;
@ -98,11 +99,8 @@ CcpUnmapCache(PVOID Context)
{
PWORK_QUEUE_WITH_CONTEXT WorkItem = (PWORK_QUEUE_WITH_CONTEXT)Context;
DPRINT("Unmapping (finally) %x\n", WorkItem->ToUnmap);
WorkItem->AcquireForLazyWrite(WorkItem->LazyContext, TRUE);
MiFlushMappedSection(WorkItem->ToUnmap, &WorkItem->FileOffset, &WorkItem->MapSize, WorkItem->Dirty);
WorkItem->ReleaseFromLazyWrite(WorkItem->LazyContext);
MmUnmapCacheViewInSystemSpace(WorkItem->ToUnmap);
MmFinalizeSegment(WorkItem->ToDeref);
ObDereferenceObject(WorkItem->ToDeref);
ExFreePool(WorkItem);
DPRINT("Done\n");
}
@ -134,9 +132,7 @@ VOID CcpDereferenceCache(ULONG Start, BOOLEAN Immediate)
if (Immediate)
{
PMM_CACHE_SECTION_SEGMENT ToDeref = Bcb->SectionObject;
BOOLEAN Dirty = Bcb->Dirty;
PROS_SECTION_OBJECT ToDeref = Bcb->SectionObject;
Bcb->Map = NULL;
Bcb->SectionObject = NULL;
Bcb->BaseAddress = NULL;
@ -147,9 +143,10 @@ VOID CcpDereferenceCache(ULONG Start, BOOLEAN Immediate)
RemoveEntryList(&Bcb->ThisFileList);
CcpUnlock();
MiFlushMappedSection(ToUnmap, &BaseOffset, &MappedSize, Dirty);
if (Dirty)
MiFlushMappedSection(ToUnmap, &BaseOffset, &MappedSize, Dirty);
MmUnmapCacheViewInSystemSpace(ToUnmap);
MmFinalizeSegment(ToDeref);
ObDereferenceObject(ToDeref);
CcpLock();
}
else
@ -186,7 +183,7 @@ VOID CcpDereferenceCache(ULONG Start, BOOLEAN Immediate)
/* Needs mutex */
ULONG CcpAllocateCacheSections
(PFILE_OBJECT FileObject,
PMM_CACHE_SECTION_SEGMENT SectionObject)
PROS_SECTION_OBJECT SectionObject)
{
ULONG i = INVALID_CACHE;
PNOCC_CACHE_MAP Map;
@ -313,13 +310,12 @@ CcpMapData
/* Note: windows 2000 drivers treat this as a bool */
//BOOLEAN Wait = (Flags & MAP_WAIT) || (Flags == TRUE);
LARGE_INTEGER Target, EndInterval;
ULONG BcbHead;
ULONG BcbHead, SectionSize, ViewSize;
PNOCC_BCB Bcb = NULL;
PMM_CACHE_SECTION_SEGMENT SectionObject = NULL;
PROS_SECTION_OBJECT SectionObject = NULL;
NTSTATUS Status;
PNOCC_CACHE_MAP Map = (PNOCC_CACHE_MAP)FileObject->SectionObjectPointer->SharedCacheMap;
ULONG SectionSize;
ULONG ViewSize = CACHE_STRIPE;
ViewSize = CACHE_STRIPE;
if (!Map)
{
@ -424,10 +420,11 @@ retry:
}
DPRINT("Selected BCB #%x\n", BcbHead);
ViewSize = CACHE_STRIPE;
Bcb = &CcCacheSections[BcbHead];
Status = MmMapCacheViewInSystemSpaceAtOffset
(SectionObject,
(SectionObject->Segment,
&Bcb->BaseAddress,
&Target,
&ViewSize);
@ -436,7 +433,7 @@ retry:
{
*BcbResult = NULL;
*Buffer = NULL;
MmFinalizeSegment(SectionObject);
ObDereferenceObject(SectionObject);
RemoveEntryList(&Bcb->ThisFileList);
RtlZeroMemory(Bcb, sizeof(*Bcb));
RtlClearBit(CcCacheBitmap, BcbHead);
@ -481,17 +478,16 @@ cleanup:
{
// Fault in the pages. This forces reads to happen now.
ULONG i;
CHAR Dummy;
PCHAR FaultIn = Bcb->BaseAddress;
DPRINT1
DPRINT
("Faulting in pages at this point: file %wZ %08x%08x:%x\n",
&FileObject->FileName,
Bcb->FileOffset.HighPart,
Bcb->FileOffset.LowPart,
Bcb->Length);
for (i = 0; i < Bcb->Length; i++)
for (i = 0; i < Bcb->Length; i += PAGE_SIZE)
{
Dummy = FaultIn[i];
FaultIn[i] ^= 0;
}
}
ASSERT(Bcb >= CcCacheSections && Bcb < (CcCacheSections + CACHE_NUM_SECTIONS));
@ -657,7 +653,7 @@ CcPreparePinWrite(IN PFILE_OBJECT FileObject,
ULONG OldProtect;
#endif
DPRINT1("CcPreparePinWrite(%x:%x)\n", Buffer, Length);
DPRINT("CcPreparePinWrite(%x:%x)\n", Buffer, Length);
Result = CcPinRead
(FileObject,

317
reactos/ntoskrnl/cache/section/data.c vendored
View File

@ -52,6 +52,8 @@
#define DPRINTC DPRINT
LIST_ENTRY MiSegmentList;
extern KEVENT MpwThreadEvent;
extern KSPIN_LOCK MiSectionPageTableLock;
@ -59,15 +61,6 @@ extern KSPIN_LOCK MiSectionPageTableLock;
ULONG_PTR MmSubsectionBase;
NTSTATUS
NTAPI
MiSimpleRead
(PFILE_OBJECT FileObject,
PLARGE_INTEGER FileOffset,
PVOID Buffer,
ULONG Length,
PIO_STATUS_BLOCK ReadStatus);
static const INFORMATION_CLASS_INFO ExSectionInfoClass[] =
{
ICI_SQ_SAME( sizeof(SECTION_BASIC_INFORMATION), sizeof(ULONG), ICIF_QUERY ), /* SectionBasicInformation */
@ -80,18 +73,21 @@ static const INFORMATION_CLASS_INFO ExSectionInfoClass[] =
VOID
NTAPI
_MmLockCacheSectionSegment(PMM_CACHE_SECTION_SEGMENT Segment, const char *file, int line)
_MmLockSectionSegment(PMM_SECTION_SEGMENT Segment, const char *file, int line)
{
DPRINT("MmLockSectionSegment(%p,%s:%d)\n", Segment, file, line);
//DPRINT("MmLockSectionSegment(%p,%s:%d)\n", Segment, file, line);
ExAcquireFastMutex(&Segment->Lock);
Segment->Locked = TRUE;
}
VOID
NTAPI
_MmUnlockCacheSectionSegment(PMM_CACHE_SECTION_SEGMENT Segment, const char *file, int line)
_MmUnlockSectionSegment(PMM_SECTION_SEGMENT Segment, const char *file, int line)
{
ASSERT(Segment->Locked);
Segment->Locked = FALSE;
ExReleaseFastMutex(&Segment->Lock);
DPRINT("MmUnlockSectionSegment(%p,%s:%d)\n", Segment, file, line);
//DPRINT("MmUnlockSectionSegment(%p,%s:%d)\n", Segment, file, line);
}
NTSTATUS
@ -104,7 +100,7 @@ MiZeroFillSection
PFN_NUMBER Page;
PMMSUPPORT AddressSpace;
PMEMORY_AREA MemoryArea;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
LARGE_INTEGER FileOffset = *FileOffsetPtr, End, FirstMapped;
DPRINT("MiZeroFillSection(Address %x,Offset %x,Length %x)\n", Address, FileOffset.LowPart, Length);
AddressSpace = MmGetKernelAddressSpace();
@ -116,11 +112,11 @@ MiZeroFillSection
return STATUS_NOT_MAPPED_DATA;
}
Segment = MemoryArea->Data.CacheData.Segment;
Segment = MemoryArea->Data.SectionData.Segment;
End.QuadPart = FileOffset.QuadPart + Length;
End.LowPart = PAGE_ROUND_DOWN(End.LowPart);
FileOffset.LowPart = PAGE_ROUND_UP(FileOffset.LowPart);
FirstMapped.QuadPart = MemoryArea->Data.CacheData.ViewOffset.QuadPart;
FirstMapped.QuadPart = MemoryArea->Data.SectionData.ViewOffset.QuadPart;
DPRINT
("Pulling zero pages for %08x%08x-%08x%08x\n",
FileOffset.u.HighPart, FileOffset.u.LowPart,
@ -134,12 +130,12 @@ MiZeroFillSection
break;
MmLockAddressSpace(AddressSpace);
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
Entry = MiGetPageEntryCacheSectionSegment(Segment, &FileOffset);
Entry = MmGetPageEntrySectionSegment(Segment, &FileOffset);
if (Entry == 0)
{
MiSetPageEntryCacheSectionSegment(Segment, &FileOffset, MAKE_PFN_SSE(Page));
MmSetPageEntrySectionSegment(Segment, &FileOffset, MAKE_PFN_SSE(Page));
Address = ((PCHAR)MemoryArea->StartingAddress) + FileOffset.QuadPart - FirstMapped.QuadPart;
MmReferencePage(Page);
MmCreateVirtualMapping(NULL, Address, PAGE_READWRITE, &Page, 1);
@ -148,7 +144,7 @@ MiZeroFillSection
else
MmReleasePageMemoryConsumer(MC_CACHE, Page);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
MmUnlockAddressSpace(AddressSpace);
FileOffset.QuadPart += PAGE_SIZE;
@ -170,7 +166,7 @@ _MiFlushMappedSection
ULONG_PTR PageAddress;
PMMSUPPORT AddressSpace = MmGetKernelAddressSpace();
PMEMORY_AREA MemoryArea;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
ULONG_PTR BeginningAddress, EndingAddress;
LARGE_INTEGER ViewOffset;
LARGE_INTEGER FileOffset;
@ -189,12 +185,12 @@ _MiFlushMappedSection
}
BeginningAddress = PAGE_ROUND_DOWN((ULONG_PTR)MemoryArea->StartingAddress);
EndingAddress = PAGE_ROUND_UP((ULONG_PTR)MemoryArea->EndingAddress);
Segment = MemoryArea->Data.CacheData.Segment;
ViewOffset.QuadPart = MemoryArea->Data.CacheData.ViewOffset.QuadPart;
Segment = MemoryArea->Data.SectionData.Segment;
ViewOffset.QuadPart = MemoryArea->Data.SectionData.ViewOffset.QuadPart;
ASSERT(ViewOffset.QuadPart == BaseOffset->QuadPart);
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
Pages = ExAllocatePool
(NonPagedPool,
@ -206,7 +202,7 @@ _MiFlushMappedSection
ASSERT(FALSE);
}
DPRINT("Getting pages in range %08x-%08x\n", BeginningAddress, EndingAddress);
//DPRINT("Getting pages in range %08x-%08x\n", BeginningAddress, EndingAddress);
for (PageAddress = BeginningAddress;
PageAddress < EndingAddress;
@ -215,34 +211,36 @@ _MiFlushMappedSection
ULONG Entry;
FileOffset.QuadPart = ViewOffset.QuadPart + PageAddress - BeginningAddress;
Entry =
MiGetPageEntryCacheSectionSegment
(MemoryArea->Data.CacheData.Segment,
MmGetPageEntrySectionSegment
(MemoryArea->Data.SectionData.Segment,
&FileOffset);
Page = PFN_FROM_SSE(Entry);
if (Entry != 0 && !IS_SWAP_FROM_SSE(Entry) &&
(MmIsDirtyPageRmap(Page) || IS_DIRTY_SSE(Entry)) &&
FileOffset.QuadPart < FileSize->QuadPart)
{
Pages[(PageAddress - BeginningAddress) >> PAGE_SHIFT] = Page;
MmReferencePage(Page);
Pages[(PageAddress - BeginningAddress) >> PAGE_SHIFT] = Entry;
}
else
Pages[(PageAddress - BeginningAddress) >> PAGE_SHIFT] = 0;
}
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
MmUnlockAddressSpace(AddressSpace);
for (PageAddress = BeginningAddress;
PageAddress < EndingAddress;
PageAddress += PAGE_SIZE)
{
ULONG Entry;
FileOffset.QuadPart = ViewOffset.QuadPart + PageAddress - BeginningAddress;
Page = Pages[(PageAddress - BeginningAddress) >> PAGE_SHIFT];
Entry = Pages[(PageAddress - BeginningAddress) >> PAGE_SHIFT];
Page = PFN_FROM_SSE(Entry);
if (Page)
{
ULONG Entry;
if (WriteData) {
DPRINT("MiWriteBackPage(%wZ,addr %x,%08x%08x)\n", &Segment->FileObject->FileName, PageAddress, FileOffset.u.HighPart, FileOffset.u.LowPart);
//DPRINT("MiWriteBackPage(%wZ,addr %x,%08x%08x)\n", &Segment->FileObject->FileName, PageAddress, FileOffset.u.HighPart, FileOffset.u.LowPart);
Status = MiWriteBackPage(Segment->FileObject, &FileOffset, PAGE_SIZE, Page);
} else
Status = STATUS_SUCCESS;
@ -250,11 +248,11 @@ _MiFlushMappedSection
if (NT_SUCCESS(Status)) {
MmLockAddressSpace(AddressSpace);
MmSetCleanAllRmaps(Page);
MmLockCacheSectionSegment(Segment);
Entry = MiGetPageEntryCacheSectionSegment(Segment, &FileOffset);
MmLockSectionSegment(Segment);
Entry = MmGetPageEntrySectionSegment(Segment, &FileOffset);
if (Entry && !IS_SWAP_FROM_SSE(Entry) && PFN_FROM_SSE(Entry) == Page)
MiSetPageEntryCacheSectionSegment(Segment, &FileOffset, CLEAN_SSE(Entry));
MmUnlockCacheSectionSegment(Segment);
MmSetPageEntrySectionSegment(Segment, &FileOffset, CLEAN_SSE(Entry));
MmUnlockSectionSegment(Segment);
MmUnlockAddressSpace(AddressSpace);
} else {
DPRINT
@ -268,6 +266,7 @@ _MiFlushMappedSection
&Segment->FileObject->FileName,
Status);
}
MmDereferencePage(Page);
}
}
@ -278,76 +277,91 @@ _MiFlushMappedSection
VOID
NTAPI
MmFinalizeSegment(PMM_CACHE_SECTION_SEGMENT Segment)
MmFinalizeSegment(PMM_SECTION_SEGMENT Segment)
{
KIRQL OldIrql = 0;
MmLockCacheSectionSegment(Segment);
DPRINT("Finalize segment %p\n", Segment);
MmLockSectionSegment(Segment);
RemoveEntryList(&Segment->ListOfSegments);
if (Segment->Flags & MM_DATAFILE_SEGMENT) {
KeAcquireSpinLock(&Segment->FileObject->IrpListLock, &OldIrql);
if (Segment->Flags & MM_SEGMENT_FINALIZE) {
KeReleaseSpinLock(&Segment->FileObject->IrpListLock, OldIrql);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return;
} else {
Segment->Flags |= MM_SEGMENT_FINALIZE;
}
}
DPRINTC("Finalizing segment %x\n", Segment);
if (Segment->Flags & MM_DATAFILE_SEGMENT)
{
//Segment->FileObject->SectionObjectPointer->DataSectionObject = NULL;
Segment->Flags |= MM_SEGMENT_FINALIZE;
DPRINTC("Finalizing data file segment %p\n", Segment);
Segment->FileObject->SectionObjectPointer->DataSectionObject = NULL;
KeReleaseSpinLock(&Segment->FileObject->IrpListLock, OldIrql);
MiFreePageTablesSectionSegment(Segment, MiFreeSegmentPage);
MmUnlockCacheSectionSegment(Segment);
MmFreePageTablesSectionSegment(Segment, MiFreeSegmentPage);
MmUnlockSectionSegment(Segment);
DPRINT("Dereference file object %wZ\n", &Segment->FileObject->FileName);
ObDereferenceObject(Segment->FileObject);
DPRINT("Done with %wZ\n", &Segment->FileObject->FileName);
Segment->FileObject = NULL;
} else {
MiFreePageTablesSectionSegment(Segment, MiFreeSegmentPage);
MmUnlockCacheSectionSegment(Segment);
DPRINTC("Finalizing segment %p\n", Segment);
MmFreePageTablesSectionSegment(Segment, MiFreeSegmentPage);
MmUnlockSectionSegment(Segment);
}
DPRINTC("Segment %x destroy\n", Segment);
DPRINTC("Segment %p destroy\n", Segment);
ExFreePoolWithTag(Segment, TAG_MM_SECTION_SEGMENT);
}
NTSTATUS
NTAPI
MmCreateCacheSection
(PMM_CACHE_SECTION_SEGMENT *SegmentObject,
ACCESS_MASK DesiredAccess,
POBJECT_ATTRIBUTES ObjectAttributes,
PLARGE_INTEGER UMaximumSize,
ULONG SectionPageProtection,
ULONG AllocationAttributes,
PFILE_OBJECT FileObject)
MmCreateCacheSection(PROS_SECTION_OBJECT *SectionObject,
ACCESS_MASK DesiredAccess,
POBJECT_ATTRIBUTES ObjectAttributes,
PLARGE_INTEGER UMaximumSize,
ULONG SectionPageProtection,
ULONG AllocationAttributes,
PFILE_OBJECT FileObject)
/*
* Create a section backed by a data file
*/
{
PROS_SECTION_OBJECT Section;
NTSTATUS Status;
ULARGE_INTEGER MaximumSize;
PMM_CACHE_SECTION_SEGMENT Segment;
ULONG FileAccess;
PMM_SECTION_SEGMENT Segment;
IO_STATUS_BLOCK Iosb;
CC_FILE_SIZES FileSizes;
FILE_STANDARD_INFORMATION FileInfo;
KIRQL OldIrql;
DPRINT("MmCreateDataFileSection\n");
/*
* Check file access required
* Create the section
*/
if (SectionPageProtection & PAGE_READWRITE ||
SectionPageProtection & PAGE_EXECUTE_READWRITE)
Status = ObCreateObject(ExGetPreviousMode(),
MmSectionObjectType,
ObjectAttributes,
ExGetPreviousMode(),
NULL,
sizeof(ROS_SECTION_OBJECT),
0,
0,
(PVOID*)(PVOID)&Section);
if (!NT_SUCCESS(Status))
{
FileAccess = FILE_READ_DATA | FILE_WRITE_DATA;
DPRINT("Failed: %x\n", Status);
return(Status);
}
else
{
FileAccess = FILE_READ_DATA;
}
/*
* Reference the file handle
* Initialize it
*/
ObReferenceObject(FileObject);
RtlZeroMemory(Section, sizeof(ROS_SECTION_OBJECT));
Section->SectionPageProtection = SectionPageProtection;
Section->AllocationAttributes = AllocationAttributes;
Section->Segment = NULL;
Section->FileObject = FileObject;
DPRINT("Getting original file size\n");
/* A hack: If we're cached, we can overcome deadlocking with the upper
@ -363,18 +377,21 @@ MmCreateCacheSection
* (as in case of the EXT2FS driver by Manoj Paul Joseph where the
* standard file information is filled on first request).
*/
DPRINT("Querying info\n");
Status = IoQueryFileInformation
(FileObject,
FileStandardInformation,
sizeof(FILE_STANDARD_INFORMATION),
&FileInfo,
&Iosb.Information);
DPRINT("Query => %x\n", Status);
if (!NT_SUCCESS(Status))
{
DPRINT("Status %x\n", Status);
ObDereferenceObject(Section);
return Status;
}
ASSERT(Status != STATUS_PENDING);
FileSizes.ValidDataLength = FileInfo.EndOfFile;
@ -385,9 +402,12 @@ MmCreateCacheSection
/*
* FIXME: Revise this once a locking order for file size changes is
* decided
*
* We're handed down a maximum size in every case. Should we still check at all?
*/
if (UMaximumSize != NULL)
if (UMaximumSize != NULL && UMaximumSize->QuadPart)
{
DPRINT("Taking maximum %x\n", UMaximumSize->LowPart);
MaximumSize.QuadPart = UMaximumSize->QuadPart;
}
else
@ -396,41 +416,98 @@ MmCreateCacheSection
MaximumSize.QuadPart = FileSizes.FileSize.QuadPart;
}
Segment = ExAllocatePoolWithTag(NonPagedPool, sizeof(MM_CACHE_SECTION_SEGMENT),
/* Mapping zero-sized files isn't allowed. */
if (MaximumSize.QuadPart == 0)
{
DPRINT("Zero size file\n");
ObDereferenceObject(Section);
return STATUS_FILE_INVALID;
}
Segment = ExAllocatePoolWithTag(NonPagedPool, sizeof(MM_SECTION_SEGMENT),
TAG_MM_SECTION_SEGMENT);
if (Segment == NULL)
{
DPRINT("Failed: STATUS_NO_MEMORY\n");
ObDereferenceObject(Section);
return(STATUS_NO_MEMORY);
}
DPRINT("Zeroing %x\n", Segment);
RtlZeroMemory(Segment, sizeof(MM_SECTION_SEGMENT));
ExInitializeFastMutex(&Segment->Lock);
Segment->ReferenceCount = 1;
Segment->Locked = TRUE;
RtlZeroMemory(&Segment->Image, sizeof(Segment->Image));
Section->Segment = Segment;
KeAcquireSpinLock(&FileObject->IrpListLock, &OldIrql);
/*
* Set the lock before assigning the segment to the file object
*/
ExAcquireFastMutex(&Segment->Lock);
DPRINT("Filling out Segment info (No previous data section)\n");
ObReferenceObject(FileObject);
Segment->FileObject = FileObject;
Segment->Protection = SectionPageProtection;
Segment->Flags = MM_DATAFILE_SEGMENT;
memset(&Segment->Image, 0, sizeof(Segment->Image));
Segment->WriteCopy = FALSE;
if (AllocationAttributes & SEC_RESERVE)
* If this file hasn't been mapped as a data file before then allocate a
* section segment to describe the data file mapping
*/
if (FileObject->SectionObjectPointer->DataSectionObject == NULL)
{
Segment->Length.QuadPart = Segment->RawLength.QuadPart = 0;
FileObject->SectionObjectPointer->DataSectionObject = (PVOID)Segment;
KeReleaseSpinLock(&FileObject->IrpListLock, OldIrql);
/*
* Set the lock before assigning the segment to the file object
*/
ExAcquireFastMutex(&Segment->Lock);
DPRINT("Filling out Segment info (No previous data section)\n");
ObReferenceObject(FileObject);
Segment->FileObject = FileObject;
Segment->Protection = SectionPageProtection;
Segment->Flags = MM_DATAFILE_SEGMENT;
memset(&Segment->Image, 0, sizeof(Segment->Image));
Segment->WriteCopy = FALSE;
if (AllocationAttributes & SEC_RESERVE)
{
Segment->Length.QuadPart = Segment->RawLength.QuadPart = 0;
}
else
{
Segment->RawLength.QuadPart = MaximumSize.QuadPart;
Segment->Length.QuadPart = PAGE_ROUND_UP(Segment->RawLength.QuadPart);
}
MiInitializeSectionPageTable(Segment);
InsertHeadList(&MiSegmentList, &Segment->ListOfSegments);
}
else
{
Segment->RawLength = MaximumSize;
Segment->Length.QuadPart = PAGE_ROUND_UP(Segment->RawLength.QuadPart);
KeReleaseSpinLock(&FileObject->IrpListLock, OldIrql);
DPRINTC("Free Segment %x\n", Segment);
ExFreePool(Segment);
DPRINT("Filling out Segment info (previous data section)\n");
/*
* If the file is already mapped as a data file then we may need
* to extend it
*/
Segment =
(PMM_SECTION_SEGMENT)FileObject->SectionObjectPointer->
DataSectionObject;
Section->Segment = Segment;
(void)InterlockedIncrementUL(&Segment->ReferenceCount);
MmLockSectionSegment(Segment);
if (MaximumSize.QuadPart > Segment->RawLength.QuadPart &&
!(AllocationAttributes & SEC_RESERVE))
{
Segment->RawLength.QuadPart = MaximumSize.QuadPart;
Segment->Length.QuadPart = PAGE_ROUND_UP(Segment->RawLength.QuadPart);
}
}
MiInitializeSectionPageTable(Segment);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
Section->MaximumSize.QuadPart = MaximumSize.QuadPart;
/* Extend file if section is longer */
DPRINT("MaximumSize %08x%08x ValidDataLength %08x%08x\n",
@ -444,14 +521,14 @@ MmCreateCacheSection
if (!NT_SUCCESS(Status))
{
DPRINT("Could not expand section\n");
ObDereferenceObject(Section);
return Status;
}
}
DPRINTC("Segment %x created (%x)\n", Segment, Segment->Flags);
*SegmentObject = Segment;
*SectionObject = Section;
return(STATUS_SUCCESS);
}
@ -459,7 +536,7 @@ NTSTATUS
NTAPI
_MiMapViewOfSegment
(PMMSUPPORT AddressSpace,
PMM_CACHE_SECTION_SEGMENT Segment,
PMM_SECTION_SEGMENT Segment,
PVOID* BaseAddress,
SIZE_T ViewSize,
ULONG Protect,
@ -494,11 +571,11 @@ _MiMapViewOfSegment
DPRINTC("MiMapViewOfSegment %x %x %x %x %x %wZ %s:%d\n", MmGetAddressSpaceOwner(AddressSpace), *BaseAddress, Segment, ViewOffset ? ViewOffset->LowPart : 0, ViewSize, Segment->FileObject ? &Segment->FileObject->FileName : NULL, file, line);
MArea->Data.CacheData.Segment = Segment;
MArea->Data.SectionData.Segment = Segment;
if (ViewOffset)
MArea->Data.CacheData.ViewOffset = *ViewOffset;
MArea->Data.SectionData.ViewOffset = *ViewOffset;
else
MArea->Data.CacheData.ViewOffset.QuadPart = 0;
MArea->Data.SectionData.ViewOffset.QuadPart = 0;
#if 0
MArea->NotPresent = MmNotPresentFaultPageFile;
@ -506,6 +583,9 @@ _MiMapViewOfSegment
MArea->PageOut = MmPageOutPageFileView;
#endif
MmInitializeRegion(&MArea->Data.SectionData.RegionListHead,
ViewSize, 0, Protect);
DPRINTC
("MiMapViewOfSegment(P %x, A %x, T %x)\n",
MmGetAddressSpaceOwner(AddressSpace), *BaseAddress, MArea->Type);
@ -516,13 +596,13 @@ _MiMapViewOfSegment
VOID
NTAPI
MiFreeSegmentPage
(PMM_CACHE_SECTION_SEGMENT Segment,
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER FileOffset)
{
ULONG Entry;
PFILE_OBJECT FileObject = Segment->FileObject;
Entry = MiGetPageEntryCacheSectionSegment(Segment, FileOffset);
Entry = MmGetPageEntrySectionSegment(Segment, FileOffset);
DPRINTC("MiFreeSegmentPage(%x:%08x%08x -> Entry %x\n",
Segment, FileOffset->HighPart, FileOffset->LowPart, Entry);
@ -537,7 +617,7 @@ MiFreeSegmentPage
}
DPRINTC("Free page %x (off %x from %x) (ref ct %d, ent %x, dirty? %s)\n", OldPage, FileOffset->LowPart, Segment, MmGetReferenceCountPage(OldPage), Entry, IS_DIRTY_SSE(Entry) ? "true" : "false");
MiSetPageEntryCacheSectionSegment(Segment, FileOffset, 0);
MmSetPageEntrySectionSegment(Segment, FileOffset, 0);
MmReleasePageMemoryConsumer(MC_CACHE, OldPage);
}
else if (IS_SWAP_FROM_SSE(Entry))
@ -558,7 +638,7 @@ MmFreeCacheSectionPage
PVOID *ContextData = Context;
PMMSUPPORT AddressSpace;
PEPROCESS Process;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
LARGE_INTEGER Offset;
DPRINT("MmFreeSectionPage(%x,%x,%x,%x,%d)\n", MmGetAddressSpaceOwner(ContextData[0]), Address, Page, SwapEntry, Dirty);
@ -568,9 +648,9 @@ MmFreeCacheSectionPage
Address = (PVOID)PAGE_ROUND_DOWN(Address);
Segment = ContextData[1];
Offset.QuadPart = (ULONG_PTR)Address - (ULONG_PTR)MemoryArea->StartingAddress +
MemoryArea->Data.CacheData.ViewOffset.QuadPart;
MemoryArea->Data.SectionData.ViewOffset.QuadPart;
Entry = MiGetPageEntryCacheSectionSegment(Segment, &Offset);
Entry = MmGetPageEntrySectionSegment(Segment, &Offset);
if (Page)
{
@ -583,7 +663,7 @@ MmFreeCacheSectionPage
if (Page != 0 && PFN_FROM_SSE(Entry) == Page && Dirty)
{
DPRINT("Freeing section page %x:%x -> %x\n", Segment, Offset.LowPart, Entry);
MiSetPageEntryCacheSectionSegment(Segment, &Offset, DIRTY_SSE(Entry));
MmSetPageEntrySectionSegment(Segment, &Offset, DIRTY_SSE(Entry));
}
else if (SwapEntry != 0)
{
@ -599,7 +679,7 @@ MmUnmapViewOfCacheSegment
{
PVOID Context[2];
PMEMORY_AREA MemoryArea;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
MemoryArea = MmLocateMemoryAreaByAddress(AddressSpace, BaseAddress);
if (MemoryArea == NULL)
@ -609,17 +689,17 @@ MmUnmapViewOfCacheSegment
}
MemoryArea->DeleteInProgress = TRUE;
Segment = MemoryArea->Data.CacheData.Segment;
MemoryArea->Data.CacheData.Segment = NULL;
Segment = MemoryArea->Data.SectionData.Segment;
MemoryArea->Data.SectionData.Segment = NULL;
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
Context[0] = AddressSpace;
Context[1] = Segment;
DPRINT("MmFreeMemoryArea(%x,%x)\n", MmGetAddressSpaceOwner(AddressSpace), MemoryArea->StartingAddress);
MmFreeMemoryArea(AddressSpace, MemoryArea, MmFreeCacheSectionPage, Context);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
DPRINTC("MiUnmapViewOfSegment %x %x %x\n", MmGetAddressSpaceOwner(AddressSpace), BaseAddress, Segment);
@ -629,16 +709,17 @@ MmUnmapViewOfCacheSegment
NTSTATUS
NTAPI
MmExtendCacheSection
(PMM_CACHE_SECTION_SEGMENT Segment,
(PROS_SECTION_OBJECT Section,
PLARGE_INTEGER NewSize,
BOOLEAN ExtendFile)
{
LARGE_INTEGER OldSize;
PMM_SECTION_SEGMENT Segment = Section->Segment;
DPRINT("Extend Segment %x\n", Segment);
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
OldSize.QuadPart = Segment->RawLength.QuadPart;
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
DPRINT("OldSize %08x%08x NewSize %08x%08x\n",
OldSize.u.HighPart, OldSize.u.LowPart,
@ -651,17 +732,17 @@ MmExtendCacheSection
if (!NT_SUCCESS(Status)) return Status;
}
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
Segment->RawLength.QuadPart = NewSize->QuadPart;
Segment->Length.QuadPart = MAX(Segment->Length.QuadPart, PAGE_ROUND_UP(Segment->RawLength.LowPart));
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return STATUS_SUCCESS;
}
NTSTATUS
NTAPI
MmMapCacheViewInSystemSpaceAtOffset
(IN PMM_CACHE_SECTION_SEGMENT Segment,
(IN PMM_SECTION_SEGMENT Segment,
OUT PVOID *MappedBase,
PLARGE_INTEGER FileOffset,
IN OUT PULONG ViewSize)
@ -674,7 +755,7 @@ MmMapCacheViewInSystemSpaceAtOffset
AddressSpace = MmGetKernelAddressSpace();
MmLockAddressSpace(AddressSpace);
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
Status = MiMapViewOfSegment
(AddressSpace,
@ -685,7 +766,7 @@ MmMapCacheViewInSystemSpaceAtOffset
FileOffset,
0);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
MmUnlockAddressSpace(AddressSpace);
return Status;

66
reactos/ntoskrnl/cache/section/fault.c vendored
View File

@ -66,7 +66,7 @@ MmNotPresentFaultCachePage
NTSTATUS Status;
PVOID PAddress;
ULONG Consumer;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
LARGE_INTEGER FileOffset, TotalOffset;
ULONG Entry;
ULONG Attributes;
@ -88,9 +88,9 @@ MmNotPresentFaultCachePage
PAddress = MM_ROUND_DOWN(Address, PAGE_SIZE);
TotalOffset.QuadPart = (ULONG_PTR)PAddress - (ULONG_PTR)MemoryArea->StartingAddress;
Segment = MemoryArea->Data.CacheData.Segment;
Segment = MemoryArea->Data.SectionData.Segment;
TotalOffset.QuadPart += MemoryArea->Data.CacheData.ViewOffset.QuadPart;
TotalOffset.QuadPart += MemoryArea->Data.SectionData.ViewOffset.QuadPart;
FileOffset = TotalOffset;
//Consumer = (Segment->Flags & MM_DATAFILE_SEGMENT) ? MC_CACHE : MC_USER;
@ -106,12 +106,12 @@ MmNotPresentFaultCachePage
/*
* Lock the segment
*/
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
/*
* Get the entry corresponding to the offset within the section
*/
Entry = MiGetPageEntryCacheSectionSegment(Segment, &TotalOffset);
Entry = MmGetPageEntrySectionSegment(Segment, &TotalOffset);
Attributes = PAGE_READONLY;
@ -125,13 +125,13 @@ MmNotPresentFaultCachePage
if (Required->State & 2)
{
DPRINT("Set in section @ %x\n", TotalOffset.LowPart);
Status = MiSetPageEntryCacheSectionSegment
Status = MmSetPageEntrySectionSegment
(Segment, &TotalOffset, Entry = MAKE_PFN_SSE(Required->Page[0]));
if (!NT_SUCCESS(Status))
{
MmReleasePageMemoryConsumer(MC_CACHE, Required->Page[0]);
}
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
MiSetPageEvent(Process, Address);
DPRINT("Status %x\n", Status);
return STATUS_MM_RESTART_OPERATION;
@ -149,7 +149,7 @@ MmNotPresentFaultCachePage
// Drop the reference for our address space ...
MmReleasePageMemoryConsumer(MC_CACHE, Required->Page[0]);
}
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
DPRINTC("XXX Set Event %x\n", Status);
MiSetPageEvent(Process, Address);
DPRINT("Status %x\n", Status);
@ -170,7 +170,7 @@ MmNotPresentFaultCachePage
}
DPRINT("XXX Set Event %x\n", Status);
MiSetPageEvent(Process, Address);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
DPRINT("Status %x\n", Status);
return Status;
}
@ -188,8 +188,8 @@ MmNotPresentFaultCachePage
Required->FileOffset = FileOffset;
Required->Amount = PAGE_SIZE;
Required->DoAcquisition = MiReadFilePage;
MiSetPageEntryCacheSectionSegment(Segment, &TotalOffset, MAKE_SWAP_SSE(MM_WAIT_ENTRY));
MmUnlockCacheSectionSegment(Segment);
MmSetPageEntrySectionSegment(Segment, &TotalOffset, MAKE_SWAP_SSE(MM_WAIT_ENTRY));
MmUnlockSectionSegment(Segment);
return STATUS_MORE_PROCESSING_REQUIRED;
}
ASSERT(FALSE);
@ -232,36 +232,34 @@ MiCowCacheSectionPage
BOOLEAN Locked,
PMM_REQUIRED_RESOURCES Required)
{
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
PFN_NUMBER NewPage, OldPage;
NTSTATUS Status;
PVOID PAddress;
LARGE_INTEGER Offset;
PEPROCESS Process = MmGetAddressSpaceOwner(AddressSpace);
DPRINT("MmAccessFaultSectionView(%x, %x, %x, %x)\n", AddressSpace, MemoryArea, Address, Locked);
Segment = MemoryArea->Data.CacheData.Segment;
Segment = MemoryArea->Data.SectionData.Segment;
/*
* Lock the segment
*/
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
/*
* Find the offset of the page
*/
PAddress = MM_ROUND_DOWN(Address, PAGE_SIZE);
Offset.QuadPart = (ULONG_PTR)PAddress - (ULONG_PTR)MemoryArea->StartingAddress +
MemoryArea->Data.CacheData.ViewOffset.QuadPart;
MemoryArea->Data.SectionData.ViewOffset.QuadPart;
#if 0 // XXX Cache sections are not CoW. For now, treat all access violations this way.
if ((!Segment->WriteCopy &&
!MemoryArea->Data.CacheData.WriteCopyView) ||
if (!Segment->WriteCopy /*&&
!MemoryArea->Data.SectionData.WriteCopyView*/ ||
Segment->Image.Characteristics & IMAGE_SCN_MEM_SHARED)
#endif
{
#if 0 // XXX Cache sections don't have regions at present, which streamlines things
#if 0
if (Region->Protect == PAGE_READWRITE ||
Region->Protect == PAGE_EXECUTE_READWRITE)
#endif
@ -272,15 +270,15 @@ MiCowCacheSectionPage
{
DPRINTC("file %wZ\n", &Segment->FileObject->FileName);
}
Entry = MiGetPageEntryCacheSectionSegment(Segment, &Offset);
Entry = MmGetPageEntrySectionSegment(Segment, &Offset);
DPRINT("Entry %x\n", Entry);
if (Entry &&
!IS_SWAP_FROM_SSE(Entry) &&
PFN_FROM_SSE(Entry) == MmGetPfnForProcess(Process, Address)) {
MiSetPageEntryCacheSectionSegment(Segment, &Offset, DIRTY_SSE(Entry));
MmSetPageEntrySectionSegment(Segment, &Offset, DIRTY_SSE(Entry));
}
MmSetPageProtect(Process, PAddress, PAGE_READWRITE);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
DPRINT("Done\n");
return STATUS_SUCCESS;
}
@ -288,7 +286,7 @@ MiCowCacheSectionPage
else
{
DPRINT("Not supposed to be writable\n");
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return STATUS_ACCESS_VIOLATION;
}
#endif
@ -300,7 +298,7 @@ MiCowCacheSectionPage
if (MmIsPageSwapEntry(Process, Address))
{
MmGetPageFileMapping(Process, Address, &SwapEntry);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
if (SwapEntry == MM_WAIT_ENTRY)
return STATUS_SUCCESS + 1; // Wait ... somebody else is getting it right now
else
@ -313,7 +311,7 @@ MiCowCacheSectionPage
Required->Line = __LINE__;
Required->DoAcquisition = MiGetOnePage;
MmCreatePageFileMapping(Process, Address, MM_WAIT_ENTRY);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return STATUS_MORE_PROCESSING_REQUIRED;
}
@ -343,13 +341,13 @@ MiCowCacheSectionPage
{
DPRINT1("MmCreateVirtualMapping failed, not out of memory\n");
ASSERT(FALSE);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return(Status);
}
MmInsertRmap(NewPage, Process, PAddress);
MmReleasePageMemoryConsumer(MC_CACHE, OldPage);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
DPRINT("Address 0x%.8X\n", Address);
return(STATUS_SUCCESS);
@ -396,6 +394,9 @@ MmpSectionAccessFaultInner
NTSTATUS Status;
BOOLEAN Locked = FromMdl;
MM_REQUIRED_RESOURCES Resources = { 0 };
WORK_QUEUE_WITH_CONTEXT Context;
RtlZeroMemory(&Context, sizeof(WORK_QUEUE_WITH_CONTEXT));
DPRINT("MmAccessFault(Mode %d, Address %x)\n", Mode, Address);
@ -479,7 +480,6 @@ MmpSectionAccessFaultInner
{
if (Thread->ActiveFaultCount > 0)
{
WORK_QUEUE_WITH_CONTEXT Context = {0};
DPRINT("Already fault handling ... going to work item (%x)\n", Address);
Context.AddressSpace = AddressSpace;
Context.MemoryArea = MemoryArea;
@ -589,8 +589,11 @@ MmNotPresentFaultCacheSectionInner
BOOLEAN Locked = FromMdl;
PMEMORY_AREA MemoryArea;
MM_REQUIRED_RESOURCES Resources = { 0 };
WORK_QUEUE_WITH_CONTEXT Context;
NTSTATUS Status = STATUS_SUCCESS;
RtlZeroMemory(&Context, sizeof(WORK_QUEUE_WITH_CONTEXT));
if (!FromMdl)
{
MmLockAddressSpace(AddressSpace);
@ -658,9 +661,8 @@ MmNotPresentFaultCacheSectionInner
}
else if (Status == STATUS_MORE_PROCESSING_REQUIRED)
{
if (Thread->ActiveFaultCount > 1)
if (Thread->ActiveFaultCount > 2)
{
WORK_QUEUE_WITH_CONTEXT Context = {0};
DPRINTC("Already fault handling ... going to work item (%x)\n", Address);
Context.AddressSpace = AddressSpace;
Context.MemoryArea = MemoryArea;

28
reactos/ntoskrnl/cache/section/io.c vendored
View File

@ -64,6 +64,9 @@ MmGetDeviceObjectForFile(IN PFILE_OBJECT FileObject)
return IoGetRelatedDeviceObject(FileObject);
}
// Note:
// This completion function is really required. Paging io completion does almost
// nothing, including freeing the mdls.
NTSTATUS
NTAPI
MiSimpleReadComplete
@ -100,9 +103,7 @@ MiSimpleRead
PLARGE_INTEGER FileOffset,
PVOID Buffer,
ULONG Length,
#ifdef __ROS_DWARF__
BOOLEAN Paging,
#endif
PIO_STATUS_BLOCK ReadStatus)
{
NTSTATUS Status;
@ -145,11 +146,7 @@ MiSimpleRead
return STATUS_NO_MEMORY;
}
#ifndef __ROS_DWARF__
Irp->Flags |= IRP_PAGING_IO | IRP_SYNCHRONOUS_PAGING_IO | IRP_NOCACHE | IRP_SYNCHRONOUS_API;
#else
Irp->Flags |= (Paging ? IRP_PAGING_IO | IRP_SYNCHRONOUS_PAGING_IO | IRP_NOCACHE : 0) | IRP_SYNCHRONOUS_API;
#endif
Irp->UserEvent = &ReadWait;
Irp->Tail.Overlay.OriginalFileObject = FileObject;
@ -159,14 +156,14 @@ MiSimpleRead
IrpSp->FileObject = FileObject;
IrpSp->CompletionRoutine = MiSimpleReadComplete;
#ifdef __ROS_DWARF__
ObReferenceObject(FileObject);
#endif
// Non paging case, the FileObject will be dereferenced at completion
if (!Paging)
ObReferenceObject(FileObject);
Status = IoCallDriver(DeviceObject, Irp);
if (Status == STATUS_PENDING)
{
DPRINT1("KeWaitForSingleObject(&ReadWait)\n");
DPRINT("KeWaitForSingleObject(&ReadWait)\n");
if (!NT_SUCCESS
(KeWaitForSingleObject
(&ReadWait,
@ -210,13 +207,13 @@ _MiSimpleWrite
ASSERT(Buffer);
ASSERT(ReadStatus);
ObReferenceObject(FileObject);
DeviceObject = MmGetDeviceObjectForFile(FileObject);
ASSERT(DeviceObject);
DPRINT
("PAGING WRITE: FileObject %p Offset %x Length %d (%s:%d)\n",
("PAGING WRITE: FileObject %x <%wZ> Offset %x Length %d (%s:%d)\n",
FileObject,
&FileObject->FileName,
FileOffset->LowPart,
Length,
File,
@ -234,7 +231,6 @@ _MiSimpleWrite
if (!Irp)
{
ObDereferenceObject(FileObject);
return STATUS_NO_MEMORY;
}
@ -252,11 +248,9 @@ _MiSimpleWrite
Status = IoCallDriver(DeviceObject, Irp);
DPRINT("Status %x\n", Status);
ObDereferenceObject(FileObject);
if (Status == STATUS_PENDING)
{
DPRINT1("KeWaitForSingleObject(&ReadWait)\n");
DPRINT("KeWaitForSingleObject(&ReadWait)\n");
if (!NT_SUCCESS
(KeWaitForSingleObject
(&ReadWait,
@ -302,7 +296,7 @@ _MiWriteBackPage
MmDeleteHyperspaceMapping(Hyperspace);
KeLowerIrql(OldIrql);
DPRINT1("MiWriteBackPage(%wZ,%08x%08x,%s:%d)\n", &FileObject->FileName, FileOffset->u.HighPart, FileOffset->u.LowPart, File, Line);
DPRINT("MiWriteBackPage(%wZ,%08x%08x,%s:%d)\n", &FileObject->FileName, FileOffset->u.HighPart, FileOffset->u.LowPart, File, Line);
Status = MiSimpleWrite
(FileObject,
FileOffset,

101
reactos/ntoskrnl/cache/section/newmm.h vendored
View File

@ -11,12 +11,11 @@
(IS_SWAP_FROM_SSE(E) && SWAPENTRY_FROM_SSE(E) == MM_WAIT_ENTRY)
#define MAKE_PFN_SSE(P) ((P) << PAGE_SHIFT)
#define SWAPENTRY_FROM_SSE(E) ((E) >> 1)
#define MAKE_SWAP_SSE(S) (((S) << 1) | 0x1)
#define MAKE_SWAP_SSE(S) (((ULONG)(S) << 1) | 0x1)
#define DIRTY_SSE(E) ((E) | 2)
#define CLEAN_SSE(E) ((E) & ~2)
#define IS_DIRTY_SSE(E) ((E) & 2)
#define MEMORY_AREA_CACHE (2)
#define MM_SEGMENT_FINALIZE (0x40000000)
#define RMAP_SEGMENT_MASK ~((ULONG_PTR)0xff)
@ -31,7 +30,7 @@
(((Consumer) == MC_USER) || \
((Consumer) == MC_CACHE))
#define SEC_CACHE (0x40000000)
#define SEC_CACHE (0x20000000)
#define MiWaitForPageEvent(Process,Address) do { \
DPRINT("MiWaitForPageEvent %x:%x #\n", Process, Address); \
@ -47,32 +46,11 @@
#define ENTRIES_PER_ELEMENT 256
extern KEVENT MmWaitPageEvent;
typedef struct _MM_CACHE_SECTION_SEGMENT
{
FAST_MUTEX Lock; /* lock which protects the page directory */
PFILE_OBJECT FileObject;
ULARGE_INTEGER RawLength; /* length of the segment which is part of the mapped file */
ULARGE_INTEGER Length; /* absolute length of the segment */
ULONG ReferenceCount;
ULONG Protection;
ULONG Flags;
BOOLEAN WriteCopy;
struct
{
LONG FileOffset; /* start offset into the file for image sections */
ULONG_PTR VirtualAddress; /* dtart offset into the address range for image sections */
ULONG Characteristics;
} Image;
RTL_GENERIC_TABLE PageTable;
} MM_CACHE_SECTION_SEGMENT, *PMM_CACHE_SECTION_SEGMENT;
typedef struct _CACHE_SECTION_PAGE_TABLE
{
LARGE_INTEGER FileOffset;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
ULONG Refcount;
ULONG PageEntries[ENTRIES_PER_ELEMENT];
} CACHE_SECTION_PAGE_TABLE, *PCACHE_SECTION_PAGE_TABLE;
@ -111,55 +89,66 @@ typedef struct _MM_REQUIRED_RESOURCES
int Line;
} MM_REQUIRED_RESOURCES, *PMM_REQUIRED_RESOURCES;
NTSTATUS
NTAPI
MmCreateCacheSection
(PROS_SECTION_OBJECT *SectionObject,
ACCESS_MASK DesiredAccess,
POBJECT_ATTRIBUTES ObjectAttributes,
PLARGE_INTEGER UMaximumSize,
ULONG SectionPageProtection,
ULONG AllocationAttributes,
PFILE_OBJECT FileObject);
PFN_NUMBER
NTAPI
MmWithdrawSectionPage
(PMM_CACHE_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, BOOLEAN *Dirty);
(PMM_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, BOOLEAN *Dirty);
NTSTATUS
NTAPI
MmFinalizeSectionPageOut
(PMM_CACHE_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, PFN_NUMBER Page,
(PMM_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, PFN_NUMBER Page,
BOOLEAN Dirty);
/* sptab.c *******************************************************************/
VOID
NTAPI
MiInitializeSectionPageTable(PMM_CACHE_SECTION_SEGMENT Segment);
MiInitializeSectionPageTable(PMM_SECTION_SEGMENT Segment);
NTSTATUS
NTAPI
_MiSetPageEntryCacheSectionSegment
(PMM_CACHE_SECTION_SEGMENT Segment,
_MmSetPageEntrySectionSegment
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER Offset,
ULONG Entry, const char *file, int line);
ULONG
NTAPI
_MiGetPageEntryCacheSectionSegment
(PMM_CACHE_SECTION_SEGMENT Segment,
_MmGetPageEntrySectionSegment
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER Offset, const char *file, int line);
#define MiSetPageEntryCacheSectionSegment(S,O,E) _MiSetPageEntryCacheSectionSegment(S,O,E,__FILE__,__LINE__)
#define MiGetPageEntryCacheSectionSegment(S,O) _MiGetPageEntryCacheSectionSegment(S,O,__FILE__,__LINE__)
#define MmSetPageEntrySectionSegment(S,O,E) _MmSetPageEntrySectionSegment(S,O,E,__FILE__,__LINE__)
#define MmGetPageEntrySectionSegment(S,O) _MmGetPageEntrySectionSegment(S,O,__FILE__,__LINE__)
typedef VOID (NTAPI *FREE_SECTION_PAGE_FUN)
(PMM_CACHE_SECTION_SEGMENT Segment,
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER Offset);
VOID
NTAPI
MiFreePageTablesSectionSegment(PMM_CACHE_SECTION_SEGMENT Segment, FREE_SECTION_PAGE_FUN FreePage);
MmFreePageTablesSectionSegment(PMM_SECTION_SEGMENT Segment, FREE_SECTION_PAGE_FUN FreePage);
/* Yields a lock */
PMM_CACHE_SECTION_SEGMENT
PMM_SECTION_SEGMENT
NTAPI
MmGetSectionAssociation(PFN_NUMBER Page, PLARGE_INTEGER Offset);
NTSTATUS
NTAPI
MmSetSectionAssociation(PFN_NUMBER Page, PMM_CACHE_SECTION_SEGMENT Segment, PLARGE_INTEGER Offset);
MmSetSectionAssociation(PFN_NUMBER Page, PMM_SECTION_SEGMENT Segment, PLARGE_INTEGER Offset);
VOID
NTAPI
@ -181,9 +170,7 @@ MiSimpleRead
PLARGE_INTEGER FileOffset,
PVOID Buffer,
ULONG Length,
#ifdef __ROS_DWARF__
BOOLEAN Paging,
#endif
PIO_STATUS_BLOCK ReadStatus);
NTSTATUS
@ -262,7 +249,7 @@ MiWriteFilePage
VOID
NTAPI
MiFreeSegmentPage
(PMM_CACHE_SECTION_SEGMENT Segment,
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER FileOffset);
NTSTATUS
@ -283,15 +270,15 @@ MmPageOutDeleteMapping(PVOID Context, PEPROCESS Process, PVOID Address);
VOID
NTAPI
_MmLockCacheSectionSegment(PMM_CACHE_SECTION_SEGMENT Segment, const char *file, int line);
_MmLockSectionSegment(PMM_SECTION_SEGMENT Segment, const char *file, int line);
#define MmLockCacheSectionSegment(x) _MmLockCacheSectionSegment(x,__FILE__,__LINE__)
#define MmLockSectionSegment(x) _MmLockSectionSegment(x,__FILE__,__LINE__)
VOID
NTAPI
_MmUnlockCacheSectionSegment(PMM_CACHE_SECTION_SEGMENT Segment, const char *file, int line);
_MmUnlockSectionSegment(PMM_SECTION_SEGMENT Segment, const char *file, int line);
#define MmUnlockCacheSectionSegment(x) _MmUnlockCacheSectionSegment(x,__FILE__,__LINE__)
#define MmUnlockSectionSegment(x) _MmUnlockSectionSegment(x,__FILE__,__LINE__)
VOID
MmFreeCacheSectionPage
@ -306,7 +293,7 @@ _MiFlushMappedSection(PVOID BaseAddress, PLARGE_INTEGER BaseOffset, PLARGE_INTEG
VOID
NTAPI
MmFinalizeSegment(PMM_CACHE_SECTION_SEGMENT Segment);
MmFinalizeSegment(PMM_SECTION_SEGMENT Segment);
VOID
NTAPI
@ -314,7 +301,7 @@ MmFreeSectionSegments(PFILE_OBJECT FileObject);
NTSTATUS NTAPI
MmMapCacheViewInSystemSpaceAtOffset
(IN PMM_CACHE_SECTION_SEGMENT Segment,
(IN PMM_SECTION_SEGMENT Segment,
OUT PVOID * MappedBase,
IN PLARGE_INTEGER ViewOffset,
IN OUT PULONG ViewSize);
@ -323,7 +310,7 @@ NTSTATUS
NTAPI
_MiMapViewOfSegment
(PMMSUPPORT AddressSpace,
PMM_CACHE_SECTION_SEGMENT Segment,
PMM_SECTION_SEGMENT Segment,
PVOID* BaseAddress,
SIZE_T ViewSize,
ULONG Protect,
@ -388,7 +375,7 @@ MiSwapInSectionPage
NTSTATUS
NTAPI
MmExtendCacheSection(PMM_CACHE_SECTION_SEGMENT Section, PLARGE_INTEGER NewSize, BOOLEAN ExtendFile);
MmExtendCacheSection(PROS_SECTION_OBJECT Section, PLARGE_INTEGER NewSize, BOOLEAN ExtendFile);
NTSTATUS
NTAPI
@ -396,17 +383,6 @@ _MiFlushMappedSection(PVOID BaseAddress, PLARGE_INTEGER BaseOffset, PLARGE_INTEG
#define MiFlushMappedSection(A,O,S,D) _MiFlushMappedSection(A,O,S,D,__FILE__,__LINE__)
NTSTATUS
NTAPI
MmCreateCacheSection
(PMM_CACHE_SECTION_SEGMENT *SegmentObject,
ACCESS_MASK DesiredAccess,
POBJECT_ATTRIBUTES ObjectAttributes,
PLARGE_INTEGER UMaximumSize,
ULONG SectionPageProtection,
ULONG AllocationAttributes,
PFILE_OBJECT FileObject);
PVOID
NTAPI
MmGetSegmentRmap(PFN_NUMBER Page, PULONG RawOffset);
@ -420,4 +396,7 @@ MmNotPresentFaultCacheSection
ULONG
NTAPI
MiCacheEvictPages(PVOID BaseAddress, ULONG Target);
MiCacheEvictPages(PMM_SECTION_SEGMENT Segment, ULONG Target);
NTSTATUS
MiRosTrimCache(ULONG Target, ULONG Priority, PULONG NrFreed);

50
reactos/ntoskrnl/cache/section/reqtools.c vendored
View File

@ -91,9 +91,11 @@ MiReadFilePage
PLARGE_INTEGER FileOffset = &RequiredResources->FileOffset;
NTSTATUS Status;
PVOID PageBuf = NULL;
PMEMORY_AREA TmpArea;
IO_STATUS_BLOCK IOSB;
PHYSICAL_ADDRESS BoundaryAddressMultiple;
PPFN_NUMBER Pages;
PMDL Mdl;
PVOID HyperMap;
BoundaryAddressMultiple.QuadPart = 0;
@ -110,44 +112,27 @@ MiReadFilePage
return Status;
}
MmLockAddressSpace(MmGetKernelAddressSpace());
Status = MmCreateMemoryArea
(MmGetKernelAddressSpace(),
MEMORY_AREA_VIRTUAL_MEMORY,
&PageBuf,
PAGE_SIZE,
PAGE_READWRITE,
&TmpArea,
FALSE,
MEM_TOP_DOWN,
BoundaryAddressMultiple);
DPRINT("Status %x, PageBuf %x\n", Status, PageBuf);
if (!NT_SUCCESS(Status))
{
DPRINT1("STATUS_NO_MEMORY: %x\n", Status);
MmUnlockAddressSpace(MmGetKernelAddressSpace());
HyperMap = MmCreateHyperspaceMapping(*Page);
Mdl = IoAllocateMdl(HyperMap, PAGE_SIZE, FALSE, FALSE, NULL);
if (!Mdl) {
MmReleasePageMemoryConsumer(RequiredResources->Consumer, *Page);
return STATUS_NO_MEMORY;
}
Status = MmCreateVirtualMapping(NULL, PageBuf, PAGE_READWRITE, Page, 1);
if (!NT_SUCCESS(Status))
{
MmFreeMemoryArea(MmGetKernelAddressSpace(), TmpArea, NULL, NULL);
MmUnlockAddressSpace(MmGetKernelAddressSpace());
MmReleasePageMemoryConsumer(RequiredResources->Consumer, *Page);
DPRINT1("Status: %x\n", Status);
return Status;
}
MmUnlockAddressSpace(MmGetKernelAddressSpace());
MmInitializeMdl(Mdl, HyperMap, PAGE_SIZE);
Pages = (PPFN_NUMBER)(Mdl + 1);
Pages[0] = *Page;
MmProbeAndLockPages(Mdl, KernelMode, IoModifyAccess);
PageBuf = MmGetSystemAddressForMdlSafe(Mdl, NormalPagePriority);
MmDeleteHyperspaceMapping(HyperMap);
Status = MiSimpleRead
(FileObject,
FileOffset,
PageBuf,
RequiredResources->Amount,
TRUE,
&IOSB);
RtlZeroMemory
((PCHAR)PageBuf+RequiredResources->Amount,
@ -155,9 +140,8 @@ MiReadFilePage
DPRINT("Read Status %x (Page %x)\n", Status, *Page);
MmLockAddressSpace(MmGetKernelAddressSpace());
MmFreeMemoryArea(MmGetKernelAddressSpace(), TmpArea, NULL, NULL);
MmUnlockAddressSpace(MmGetKernelAddressSpace());
MmUnlockPages(Mdl);
IoFreeMdl(Mdl);
if (!NT_SUCCESS(Status))
{

66
reactos/ntoskrnl/cache/section/sptab.c vendored
View File

@ -43,7 +43,7 @@ MiSectionPageTableAllocate(PRTL_GENERIC_TABLE Table, CLONG Bytes)
{
PVOID Result;
Result = ExAllocatePoolWithTag(NonPagedPool, Bytes, 'MmPt');
DPRINT("MiSectionPageTableAllocate(%d) => %p\n", Bytes, Result);
//DPRINT("MiSectionPageTableAllocate(%d) => %p\n", Bytes, Result);
return Result;
}
@ -52,7 +52,7 @@ VOID
NTAPI
MiSectionPageTableFree(PRTL_GENERIC_TABLE Table, PVOID Data)
{
DPRINT("MiSectionPageTableFree(%p)\n", Data);
//DPRINT("MiSectionPageTableFree(%p)\n", Data);
ExFreePoolWithTag(Data, 'MmPt');
}
@ -65,6 +65,7 @@ MiSectionPageTableCompare(PRTL_GENERIC_TABLE Table, PVOID PtrA, PVOID PtrB)
BOOLEAN Result = (A->QuadPart < B->QuadPart) ? GenericLessThan :
(A->QuadPart == B->QuadPart) ? GenericEqual : GenericGreaterThan;
#if 0
DPRINT
("Compare: %08x%08x vs %08x%08x => %s\n",
A->u.HighPart, A->u.LowPart,
@ -72,6 +73,7 @@ MiSectionPageTableCompare(PRTL_GENERIC_TABLE Table, PVOID PtrA, PVOID PtrB)
Result == GenericLessThan ? "GenericLessThan" :
Result == GenericGreaterThan ? "GenericGreaterThan" :
"GenericEqual");
#endif
return Result;
}
@ -103,11 +105,13 @@ MiSectionPageTableGetOrAllocate
PLARGE_INTEGER FileOffset)
{
LARGE_INTEGER SearchFileOffset;
CACHE_SECTION_PAGE_TABLE SectionZeroPageTable;
PCACHE_SECTION_PAGE_TABLE PageTableSlice =
MiSectionPageTableGet(Table, FileOffset);
// Please zero memory when taking away zero initialization.
RtlZeroMemory(&SectionZeroPageTable, sizeof(CACHE_SECTION_PAGE_TABLE));
if (!PageTableSlice)
{
CACHE_SECTION_PAGE_TABLE SectionZeroPageTable = {0};
SearchFileOffset.QuadPart = ROUND_DOWN(FileOffset->QuadPart, ENTRIES_PER_ELEMENT * PAGE_SIZE);
SectionZeroPageTable.FileOffset = SearchFileOffset;
SectionZeroPageTable.Refcount = 1;
@ -125,7 +129,7 @@ MiSectionPageTableGetOrAllocate
VOID
NTAPI
MiInitializeSectionPageTable(PMM_CACHE_SECTION_SEGMENT Segment)
MiInitializeSectionPageTable(PMM_SECTION_SEGMENT Segment)
{
RtlInitializeGenericTable
(&Segment->PageTable,
@ -138,8 +142,8 @@ MiInitializeSectionPageTable(PMM_CACHE_SECTION_SEGMENT Segment)
NTSTATUS
NTAPI
_MiSetPageEntryCacheSectionSegment
(PMM_CACHE_SECTION_SEGMENT Segment,
_MmSetPageEntrySectionSegment
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER Offset,
ULONG Entry,
const char *file,
@ -147,6 +151,9 @@ _MiSetPageEntryCacheSectionSegment
{
ULONG PageIndex, OldEntry;
PCACHE_SECTION_PAGE_TABLE PageTable;
ASSERT(Segment->Locked);
if (Entry && !IS_SWAP_FROM_SSE(Entry))
MmGetRmapListHeadPage(PFN_FROM_SSE(Entry));
PageTable =
MiSectionPageTableGetOrAllocate(&Segment->PageTable, Offset);
if (!PageTable) return STATUS_NO_MEMORY;
@ -155,23 +162,36 @@ _MiSetPageEntryCacheSectionSegment
PageIndex =
(Offset->QuadPart - PageTable->FileOffset.QuadPart) / PAGE_SIZE;
OldEntry = PageTable->PageEntries[PageIndex];
if (OldEntry && !IS_SWAP_FROM_SSE(OldEntry)) {
MmDeleteSectionAssociation(PFN_FROM_SSE(OldEntry));
}
if (Entry && !IS_SWAP_FROM_SSE(Entry)) {
DPRINT("MiSetPageEntrySectionSegment(%p,%08x%08x,%x=>%x)\n",
Segment, Offset->u.HighPart, Offset->u.LowPart, OldEntry, Entry);
if (PFN_FROM_SSE(Entry) == PFN_FROM_SSE(OldEntry)) {
// Nothing
} else if (Entry && !IS_SWAP_FROM_SSE(Entry)) {
ASSERT(!OldEntry || IS_SWAP_FROM_SSE(OldEntry));
MmSetSectionAssociation(PFN_FROM_SSE(Entry), Segment, Offset);
} else if (OldEntry && !IS_SWAP_FROM_SSE(OldEntry)) {
ASSERT(!Entry || IS_SWAP_FROM_SSE(Entry));
MmDeleteSectionAssociation(PFN_FROM_SSE(OldEntry));
} else if (IS_SWAP_FROM_SSE(Entry)) {
ASSERT(!IS_SWAP_FROM_SSE(OldEntry));
if (OldEntry)
MmDeleteSectionAssociation(PFN_FROM_SSE(OldEntry));
} else if (IS_SWAP_FROM_SSE(OldEntry)) {
ASSERT(!IS_SWAP_FROM_SSE(Entry));
if (Entry)
MmSetSectionAssociation(PFN_FROM_SSE(OldEntry), Segment, Offset);
} else {
// We should not be replacing a page like this
ASSERT(FALSE);
}
PageTable->PageEntries[PageIndex] = Entry;
DPRINT
("MiSetPageEntrySectionSegment(%p,%08x%08x,%x) %s:%d\n",
Segment, Offset->u.HighPart, Offset->u.LowPart, Entry, file, line);
return STATUS_SUCCESS;
}
ULONG
NTAPI
_MiGetPageEntryCacheSectionSegment
(PMM_CACHE_SECTION_SEGMENT Segment,
_MmGetPageEntrySectionSegment
(PMM_SECTION_SEGMENT Segment,
PLARGE_INTEGER Offset,
const char *file,
int line)
@ -180,6 +200,7 @@ _MiGetPageEntryCacheSectionSegment
ULONG PageIndex, Result;
PCACHE_SECTION_PAGE_TABLE PageTable;
ASSERT(Segment->Locked);
FileOffset.QuadPart =
ROUND_DOWN(Offset->QuadPart, ENTRIES_PER_ELEMENT * PAGE_SIZE);
PageTable = MiSectionPageTableGet(&Segment->PageTable, &FileOffset);
@ -201,14 +222,15 @@ _MiGetPageEntryCacheSectionSegment
VOID
NTAPI
MiFreePageTablesSectionSegment
(PMM_CACHE_SECTION_SEGMENT Segment, FREE_SECTION_PAGE_FUN FreePage)
MmFreePageTablesSectionSegment
(PMM_SECTION_SEGMENT Segment, FREE_SECTION_PAGE_FUN FreePage)
{
PCACHE_SECTION_PAGE_TABLE Element;
DPRINT("MiFreePageTablesSectionSegment(%p)\n", &Segment->PageTable);
while ((Element = RtlGetElementGenericTable(&Segment->PageTable, 0))) {
DPRINT
("Delete table for %x -> %08x%08x\n",
("Delete table for <%wZ> %x -> %08x%08x\n",
Segment->FileObject ? &Segment->FileObject->FileName : NULL,
Segment,
Element->FileOffset.u.HighPart,
Element->FileOffset.u.LowPart);
@ -223,7 +245,7 @@ MiFreePageTablesSectionSegment
Entry = Element->PageEntries[i];
if (Entry && !IS_SWAP_FROM_SSE(Entry))
{
DPRINTC("Freeing page %x:%x @ %x\n", Segment, Entry, Offset.LowPart);
DPRINT("Freeing page %x:%x @ %x\n", Segment, Entry, Offset.LowPart);
FreePage(Segment, &Offset);
}
}
@ -234,12 +256,12 @@ MiFreePageTablesSectionSegment
DPRINT("Done\n");
}
PMM_CACHE_SECTION_SEGMENT
PMM_SECTION_SEGMENT
NTAPI
MmGetSectionAssociation(PFN_NUMBER Page, PLARGE_INTEGER Offset)
{
ULONG RawOffset;
PMM_CACHE_SECTION_SEGMENT Segment = NULL;
PMM_SECTION_SEGMENT Segment = NULL;
PCACHE_SECTION_PAGE_TABLE PageTable;
PageTable = (PCACHE_SECTION_PAGE_TABLE)MmGetSegmentRmap(Page, &RawOffset);
@ -254,7 +276,7 @@ MmGetSectionAssociation(PFN_NUMBER Page, PLARGE_INTEGER Offset)
NTSTATUS
NTAPI
MmSetSectionAssociation(PFN_NUMBER Page, PMM_CACHE_SECTION_SEGMENT Segment, PLARGE_INTEGER Offset)
MmSetSectionAssociation(PFN_NUMBER Page, PMM_SECTION_SEGMENT Segment, PLARGE_INTEGER Offset)
{
PCACHE_SECTION_PAGE_TABLE PageTable;
ULONG ActualOffset;

147
reactos/ntoskrnl/cache/section/swapout.c vendored
View File

@ -55,19 +55,19 @@
extern KEVENT MmWaitPageEvent;
extern FAST_MUTEX RmapListLock;
FAST_MUTEX GlobalPageOperation;
FAST_MUTEX MiGlobalPageOperation;
PFN_NUMBER
NTAPI
MmWithdrawSectionPage
(PMM_CACHE_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, BOOLEAN *Dirty)
(PMM_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, BOOLEAN *Dirty)
{
ULONG Entry;
DPRINT("MmWithdrawSectionPage(%x,%08x%08x,%x)\n", Segment, FileOffset->HighPart, FileOffset->LowPart, Dirty);
MmLockCacheSectionSegment(Segment);
Entry = MiGetPageEntryCacheSectionSegment(Segment, FileOffset);
MmLockSectionSegment(Segment);
Entry = MmGetPageEntrySectionSegment(Segment, FileOffset);
*Dirty = !!IS_DIRTY_SSE(Entry);
@ -76,28 +76,28 @@ MmWithdrawSectionPage
if (!Entry)
{
DPRINT("Stoeled!\n");
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return 0;
}
else if (MM_IS_WAIT_PTE(Entry))
{
DPRINT("WAIT\n");
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return MM_WAIT_ENTRY;
}
else if (Entry && !IS_SWAP_FROM_SSE(Entry))
{
DPRINT("Page %x\n", PFN_FROM_SSE(Entry));
*Dirty |= (Entry & 2);
MiSetPageEntryCacheSectionSegment(Segment, FileOffset, MAKE_SWAP_SSE(MM_WAIT_ENTRY));
MmUnlockCacheSectionSegment(Segment);
MmSetPageEntrySectionSegment(Segment, FileOffset, MAKE_SWAP_SSE(MM_WAIT_ENTRY));
MmUnlockSectionSegment(Segment);
return PFN_FROM_SSE(Entry);
}
else
{
DPRINT1("SWAP ENTRY?! (%x:%08x%08x)\n", Segment, FileOffset->HighPart, FileOffset->LowPart);
ASSERT(FALSE);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
return 0;
}
}
@ -105,14 +105,14 @@ MmWithdrawSectionPage
NTSTATUS
NTAPI
MmFinalizeSectionPageOut
(PMM_CACHE_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, PFN_NUMBER Page,
(PMM_SECTION_SEGMENT Segment, PLARGE_INTEGER FileOffset, PFN_NUMBER Page,
BOOLEAN Dirty)
{
NTSTATUS Status = STATUS_SUCCESS;
BOOLEAN WriteZero = FALSE, WritePage = FALSE;
SWAPENTRY Swap = MmGetSavedSwapEntryPage(Page);
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
(void)InterlockedIncrementUL(&Segment->ReferenceCount);
if (Dirty)
@ -131,7 +131,7 @@ MmFinalizeSectionPageOut
DPRINT("Status %x\n", Status);
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
if (WritePage)
{
@ -139,17 +139,17 @@ MmFinalizeSectionPageOut
Status = MiWriteBackPage(Segment->FileObject, FileOffset, PAGE_SIZE, Page);
}
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
if (WriteZero && NT_SUCCESS(Status))
{
DPRINT("Setting page entry in segment %x:%x to swap %x\n", Segment, FileOffset->LowPart, Swap);
MiSetPageEntryCacheSectionSegment(Segment, FileOffset, Swap ? MAKE_SWAP_SSE(Swap) : 0);
MmSetPageEntrySectionSegment(Segment, FileOffset, Swap ? MAKE_SWAP_SSE(Swap) : 0);
}
else
{
DPRINT("Setting page entry in segment %x:%x to page %x\n", Segment, FileOffset->LowPart, Page);
MiSetPageEntryCacheSectionSegment
MmSetPageEntrySectionSegment
(Segment, FileOffset, Page ? (Dirty ? DIRTY_SSE(MAKE_PFN_SSE(Page)) : MAKE_PFN_SSE(Page)) : 0);
}
@ -164,7 +164,7 @@ MmFinalizeSectionPageOut
MmDereferencePage(Page);
}
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
if (InterlockedDecrementUL(&Segment->ReferenceCount) == 0)
{
@ -191,24 +191,24 @@ MmPageOutCacheSection
BOOLEAN Dirty = FALSE;
PEPROCESS Process = MmGetAddressSpaceOwner(AddressSpace);
LARGE_INTEGER TotalOffset;
PMM_CACHE_SECTION_SEGMENT Segment;
PMM_SECTION_SEGMENT Segment;
PVOID PAddress = MM_ROUND_DOWN(Address, PAGE_SIZE);
TotalOffset.QuadPart = (ULONG_PTR)PAddress - (ULONG_PTR)MemoryArea->StartingAddress +
MemoryArea->Data.CacheData.ViewOffset.QuadPart;
MemoryArea->Data.SectionData.ViewOffset.QuadPart;
Segment = MemoryArea->Data.CacheData.Segment;
Segment = MemoryArea->Data.SectionData.Segment;
MmLockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
ASSERT(KeGetCurrentIrql() <= APC_LEVEL);
Dirty = MmIsDirtyPageRmap(Required->Page[0]);
Entry = MiGetPageEntryCacheSectionSegment(Segment, &TotalOffset);
Entry = MmGetPageEntrySectionSegment(Segment, &TotalOffset);
if (Dirty)
{
PFN_NUMBER OurPage;
MiSetPageEntryCacheSectionSegment(Segment, &TotalOffset, DIRTY_SSE(Entry));
MmSetPageEntrySectionSegment(Segment, &TotalOffset, DIRTY_SSE(Entry));
MmDeleteRmap(Required->Page[0], Process, Address);
MmDeleteVirtualMapping(Process, Address, FALSE, &Dirty, &OurPage);
ASSERT(OurPage == Required->Page[0]);
@ -226,7 +226,7 @@ MmPageOutCacheSection
MmDereferencePage(Required->Page[0]);
}
MmUnlockCacheSectionSegment(Segment);
MmUnlockSectionSegment(Segment);
MiSetPageEvent(Process, Address);
return Status;
}
@ -238,7 +238,7 @@ MmpPageOutPhysicalAddress(PFN_NUMBER Page)
BOOLEAN ProcRef = FALSE;
PFN_NUMBER SectionPage = 0;
PMM_RMAP_ENTRY entry;
PMM_CACHE_SECTION_SEGMENT Segment = NULL;
PMM_SECTION_SEGMENT Segment = NULL;
LARGE_INTEGER FileOffset;
PMEMORY_AREA MemoryArea;
PMMSUPPORT AddressSpace = MmGetKernelAddressSpace();
@ -250,17 +250,17 @@ MmpPageOutPhysicalAddress(PFN_NUMBER Page)
DPRINTC("Page out %x (ref ct %x)\n", Page, MmGetReferenceCountPage(Page));
ExAcquireFastMutex(&GlobalPageOperation);
ExAcquireFastMutex(&MiGlobalPageOperation);
if ((Segment = MmGetSectionAssociation(Page, &FileOffset)))
{
DPRINT1("Withdrawing page (%x) %x:%x\n", Page, Segment, FileOffset.LowPart);
DPRINTC("Withdrawing page (%x) %x:%x\n", Page, Segment, FileOffset.LowPart);
SectionPage = MmWithdrawSectionPage(Segment, &FileOffset, &Dirty);
DPRINTC("SectionPage %x\n", SectionPage);
if (SectionPage == MM_WAIT_ENTRY || SectionPage == 0)
{
DPRINT1("In progress page out %x\n", SectionPage);
ExReleaseFastMutex(&GlobalPageOperation);
ExReleaseFastMutex(&MiGlobalPageOperation);
return STATUS_UNSUCCESSFUL;
}
else
@ -434,9 +434,9 @@ bail:
DPRINTC
("Failed to page out %x, replacing %x at %x in segment %x\n",
SectionPage, FileOffset.LowPart, Segment);
MmLockCacheSectionSegment(Segment);
MiSetPageEntryCacheSectionSegment(Segment, &FileOffset, Dirty ? MAKE_PFN_SSE(Page) : DIRTY_SSE(MAKE_PFN_SSE(Page)));
MmUnlockCacheSectionSegment(Segment);
MmLockSectionSegment(Segment);
MmSetPageEntrySectionSegment(Segment, &FileOffset, Dirty ? MAKE_PFN_SSE(Page) : DIRTY_SSE(MAKE_PFN_SSE(Page)));
MmUnlockSectionSegment(Segment);
}
// Alas, we had the last reference
@ -450,60 +450,71 @@ bail:
ObDereferenceObject(Process);
}
ExReleaseFastMutex(&GlobalPageOperation);
ExReleaseFastMutex(&MiGlobalPageOperation);
DPRINTC("%s %x %x\n", NT_SUCCESS(Status) ? "Evicted" : "Spared", Page, Status);
return NT_SUCCESS(Status) ? STATUS_SUCCESS : STATUS_UNSUCCESSFUL;
}
ULONG
NTAPI
MiCacheEvictPages(PVOID BaseAddress, ULONG Target)
MiCacheEvictPages(PMM_SECTION_SEGMENT Segment, ULONG Target)
{
ULONG i, Entry, Result = 0;
ULONG Entry, Result = 0, i, j;
NTSTATUS Status;
PFN_NUMBER Page;
PMEMORY_AREA MemoryArea;
LARGE_INTEGER Offset;
PMM_CACHE_SECTION_SEGMENT Segment;
MmLockAddressSpace(MmGetKernelAddressSpace());
MemoryArea = MmLocateMemoryAreaByAddress
(MmGetKernelAddressSpace(),
BaseAddress);
MmLockSectionSegment(Segment);
ASSERT(MemoryArea);
ASSERT(MemoryArea->Type == MEMORY_AREA_CACHE);
Segment = MemoryArea->Data.CacheData.Segment;
ASSERT(Segment);
MmLockCacheSectionSegment(Segment);
for (i = 0;
i < ((ULONG_PTR)MemoryArea->EndingAddress) -
((ULONG_PTR)MemoryArea->StartingAddress) &&
Result < Target;
i += PAGE_SIZE) {
Offset.QuadPart = MemoryArea->Data.CacheData.ViewOffset.QuadPart + i;
Entry = MiGetPageEntryCacheSectionSegment(Segment, &Offset);
if (Entry && !IS_SWAP_FROM_SSE(Entry)) {
Page = PFN_FROM_SSE(Entry);
MmReferencePage(Page);
MmUnlockCacheSectionSegment(Segment);
MmUnlockAddressSpace(MmGetKernelAddressSpace());
Status = MmpPageOutPhysicalAddress(Page);
if (NT_SUCCESS(Status))
Result++;
MmLockCacheSectionSegment(Segment);
MmLockAddressSpace(MmGetKernelAddressSpace());
MmReleasePageMemoryConsumer(MC_CACHE, Page);
for (i = 0; i < RtlNumberGenericTableElements(&Segment->PageTable); i++) {
PCACHE_SECTION_PAGE_TABLE Element = RtlGetElementGenericTable(&Segment->PageTable, i);
ASSERT(Element);
Offset = Element->FileOffset;
for (j = 0; j < ENTRIES_PER_ELEMENT; j++, Offset.QuadPart += PAGE_SIZE) {
Entry = MmGetPageEntrySectionSegment(Segment, &Offset);
if (Entry && !IS_SWAP_FROM_SSE(Entry)) {
Page = PFN_FROM_SSE(Entry);
MmReferencePage(Page);
MmUnlockSectionSegment(Segment);
Status = MmpPageOutPhysicalAddress(Page);
if (NT_SUCCESS(Status))
Result++;
MmLockSectionSegment(Segment);
MmReleasePageMemoryConsumer(MC_CACHE, Page);
}
}
}
MmUnlockCacheSectionSegment(Segment);
MmUnlockAddressSpace(MmGetKernelAddressSpace());
MmUnlockSectionSegment(Segment);
return Result;
}
extern LIST_ENTRY MiSegmentList;
// Interact with legacy balance manager for now
// This can fall away when our section implementation supports
// demand paging properly
NTSTATUS
MiRosTrimCache(ULONG Target, ULONG Priority, PULONG NrFreed)
{
ULONG Freed;
PLIST_ENTRY Entry;
PMM_SECTION_SEGMENT Segment;
*NrFreed = 0;
for (Entry = MiSegmentList.Flink; *NrFreed < Target && Entry != &MiSegmentList; Entry = Entry->Flink) {
Segment = CONTAINING_RECORD(Entry, MM_SECTION_SEGMENT, ListOfSegments);
// Defer to MM to try recovering pages from it
Freed = MiCacheEvictPages(Segment, Target);
*NrFreed += Freed;
}
if (!IsListEmpty(&MiSegmentList)) {
Entry = MiSegmentList.Flink;
RemoveEntryList(Entry);
InsertTailList(&MiSegmentList, Entry);
}
return STATUS_SUCCESS;
}

18
reactos/ntoskrnl/cc/fs.c
View File

@ -248,3 +248,21 @@ CcUninitializeCacheMap (
return NT_SUCCESS(CcRosReleaseFileCache(FileObject));
#endif
}
BOOLEAN
NTAPI
CcGetFileSizes
(IN PFILE_OBJECT FileObject,
IN PCC_FILE_SIZES FileSizes)
{
PBCB Bcb;
Bcb = FileObject->SectionObjectPointer->SharedCacheMap;
if (!Bcb)
return FALSE;
FileSizes->AllocationSize = Bcb->AllocationSize;
FileSizes->FileSize = FileSizes->ValidDataLength = Bcb->FileSize;
return TRUE;
}

46
reactos/ntoskrnl/include/internal/mm.h
View File

@ -74,6 +74,7 @@ typedef ULONG_PTR SWAPENTRY;
#endif
#define MEMORY_AREA_SECTION_VIEW (1)
#define MEMORY_AREA_CACHE (2)
#define MEMORY_AREA_VIRTUAL_MEMORY (8)
#define MEMORY_AREA_OWNED_BY_ARM3 (15)
#define MEMORY_AREA_STATIC (0x80000000)
@ -193,29 +194,28 @@ typedef ULONG_PTR SWAPENTRY;
#define InterlockedExchangePte(PointerPte, Value) \
InterlockedExchange((PLONG)(PointerPte), Value)
typedef struct
{
ULONG Entry[NR_SECTION_PAGE_ENTRIES];
} SECTION_PAGE_TABLE, *PSECTION_PAGE_TABLE;
typedef struct
{
PSECTION_PAGE_TABLE PageTables[NR_SECTION_PAGE_TABLES];
} SECTION_PAGE_DIRECTORY, *PSECTION_PAGE_DIRECTORY;
typedef struct _MM_SECTION_SEGMENT
{
LONG FileOffset; /* start offset into the file for image sections */
ULONG_PTR VirtualAddress; /* dtart offset into the address range for image sections */
ULONG RawLength; /* length of the segment which is part of the mapped file */
SIZE_T Length; /* absolute length of the segment */
ULONG Protection;
FAST_MUTEX Lock; /* lock which protects the page directory */
PFILE_OBJECT FileObject;
LARGE_INTEGER RawLength; /* length of the segment which is part of the mapped file */
LARGE_INTEGER Length; /* absolute length of the segment */
ULONG ReferenceCount;
SECTION_PAGE_DIRECTORY PageDirectory;
ULONG CacheCount;
ULONG Protection;
ULONG Flags;
ULONG Characteristics;
BOOLEAN WriteCopy;
BOOLEAN Locked;
struct
{
LONG FileOffset; /* start offset into the file for image sections */
ULONG_PTR VirtualAddress; /* dtart offset into the address range for image sections */
ULONG Characteristics;
} Image;
LIST_ENTRY ListOfSegments;
RTL_GENERIC_TABLE PageTable;
} MM_SECTION_SEGMENT, *PMM_SECTION_SEGMENT;
typedef struct _MM_IMAGE_SECTION_OBJECT
@ -250,8 +250,6 @@ typedef struct _ROS_SECTION_OBJECT
};
} ROS_SECTION_OBJECT, *PROS_SECTION_OBJECT;
struct _MM_CACHE_SECTION_SEGMENT;
typedef struct _MEMORY_AREA
{
PVOID StartingAddress;
@ -270,15 +268,10 @@ typedef struct _MEMORY_AREA
struct
{
ROS_SECTION_OBJECT* Section;
ULONG ViewOffset;
LARGE_INTEGER ViewOffset;
PMM_SECTION_SEGMENT Segment;
LIST_ENTRY RegionListHead;
} SectionData;
struct
{
LARGE_INTEGER ViewOffset;
struct _MM_CACHE_SECTION_SEGMENT *Segment;
} CacheData;
struct
{
LIST_ENTRY RegionListHead;
@ -1655,7 +1648,8 @@ NTAPI
MmNotPresentFaultSectionView(
PMMSUPPORT AddressSpace,
MEMORY_AREA* MemoryArea,
PVOID Address
PVOID Address,
BOOLEAN Locked
);
NTSTATUS

4
reactos/ntoskrnl/mm/amd64/page.c
View File

@ -617,6 +617,8 @@ MmCreateVirtualMappingUnsafe(
ULONG i;
MMPTE TmplPte, *Pte;
ASSERT((ULONG_PTR)Address % PAGE_SIZE == 0);
/* Check if the range is valid */
if ((Process == NULL && Address < MmSystemRangeStart) ||
(Process != NULL && Address > MmHighestUserAddress))
@ -667,6 +669,8 @@ MmCreateVirtualMapping(PEPROCESS Process,
{
ULONG i;
ASSERT((ULONG_PTR)Address % PAGE_SIZE == 0);
for (i = 0; i < PageCount; i++)
{
if (!MmIsPageInUse(Pages[i]))

1
reactos/ntoskrnl/mm/i386/page.c
View File

@ -844,6 +844,7 @@ MmCreateVirtualMapping(PEPROCESS Process,
{
ULONG i;
ASSERT((ULONG_PTR)Address % PAGE_SIZE == 0);
for (i = 0; i < PageCount; i++)
{
if (!MmIsPageInUse(Pages[i]))

12
reactos/ntoskrnl/mm/marea.c
View File

@ -43,6 +43,7 @@
#include <ntoskrnl.h>
#define NDEBUG
#include "../cache/section/newmm.h"
#include <debug.h>
#include "ARM3/miarm.h"
@ -378,7 +379,7 @@ MmInsertMemoryArea(
{
PMMVAD Vad;
ASSERT(marea->Type == MEMORY_AREA_VIRTUAL_MEMORY || marea->Type == MEMORY_AREA_SECTION_VIEW);
ASSERT(marea->Type == MEMORY_AREA_VIRTUAL_MEMORY || marea->Type == MEMORY_AREA_SECTION_VIEW || marea->Type == MEMORY_AREA_CACHE);
Vad = ExAllocatePoolWithTag(NonPagedPool, sizeof(MMVAD), TAG_MVAD);
ASSERT(Vad);
RtlZeroMemory(Vad, sizeof(MMVAD));
@ -771,7 +772,7 @@ MmFreeMemoryArea(
if (MemoryArea->Vad)
{
ASSERT(MemoryArea->EndingAddress < MmSystemRangeStart);
ASSERT(MemoryArea->Type == MEMORY_AREA_VIRTUAL_MEMORY || MemoryArea->Type == MEMORY_AREA_SECTION_VIEW);
ASSERT(MemoryArea->Type == MEMORY_AREA_VIRTUAL_MEMORY || MemoryArea->Type == MEMORY_AREA_SECTION_VIEW || MemoryArea->Type == MEMORY_AREA_CACHE);
/* MmCleanProcessAddressSpace might have removed it (and this would be MmDeleteProcessAdressSpace) */
ASSERT(((PMMVAD)MemoryArea->Vad)->u.VadFlags.Spare != 0);
@ -1051,6 +1052,13 @@ MmDeleteProcessAddressSpace(PEPROCESS Process)
MmLockAddressSpace(&Process->Vm);
break;
case MEMORY_AREA_CACHE:
Address = (PVOID)MemoryArea->StartingAddress;
MmUnlockAddressSpace(&Process->Vm);
MmUnmapViewOfCacheSegment(&Process->Vm, Address);
MmLockAddressSpace(&Process->Vm);
break;
case MEMORY_AREA_VIRTUAL_MEMORY:
MmFreeVirtualMemory(Process, MemoryArea);
break;

13
reactos/ntoskrnl/mm/mmfault.c
View File

@ -9,9 +9,7 @@
/* INCLUDES *******************************************************************/
#include <ntoskrnl.h>
#ifdef NEWCC
#include "../cache/section/newmm.h"
#endif
#define NDEBUG
#include <debug.h>
@ -86,17 +84,15 @@ MmpAccessFault(KPROCESSOR_MODE Mode,
Status = STATUS_ACCESS_VIOLATION;
break;
#ifdef NEWCC
case MEMORY_AREA_CACHE:
// This code locks for itself to keep from having to break a lock
// passed in.
if (!FromMdl)
MmUnlockAddressSpace(AddressSpace);
Status = MmAccessFaultCacheSection(Mode, Address, Locked);
Status = MmAccessFaultCacheSection(Mode, Address, FromMdl);
if (!FromMdl)
MmLockAddressSpace(AddressSpace);
break;
#endif
default:
Status = STATUS_ACCESS_VIOLATION;
@ -176,7 +172,8 @@ MmNotPresentFault(KPROCESSOR_MODE Mode,
case MEMORY_AREA_SECTION_VIEW:
Status = MmNotPresentFaultSectionView(AddressSpace,
MemoryArea,
(PVOID)Address);
(PVOID)Address,
FromMdl);
break;
case MEMORY_AREA_VIRTUAL_MEMORY:
@ -185,17 +182,15 @@ MmNotPresentFault(KPROCESSOR_MODE Mode,
(PVOID)Address);
break;
#ifdef NEWCC
case MEMORY_AREA_CACHE:
// This code locks for itself to keep from having to break a lock
// passed in.
if (!FromMdl)
MmUnlockAddressSpace(AddressSpace);
Status = MmNotPresentFaultCacheSection(Mode, Address, Locked);
Status = MmNotPresentFaultCacheSection(Mode, Address, FromMdl);
if (!FromMdl)
MmLockAddressSpace(AddressSpace);
break;
#endif
default:
Status = STATUS_ACCESS_VIOLATION;

13
reactos/ntoskrnl/mm/mminit.c
View File

@ -31,6 +31,11 @@ UCHAR MmDisablePagingExecutive = 1; // Forced to off
PMMPTE MmSharedUserDataPte;
PMMSUPPORT MmKernelAddressSpace;
extern KEVENT MmWaitPageEvent;
extern FAST_MUTEX MiGlobalPageOperation;
extern LIST_ENTRY MiSegmentList;
extern NTSTATUS MiRosTrimCache(ULONG Target, ULONG Priority, PULONG NrFreed);
/* PRIVATE FUNCTIONS *********************************************************/
VOID
@ -394,6 +399,14 @@ MmInitSystem(IN ULONG Phase,
/* Initialize the kernel address space */
ASSERT(Phase == 1);
InitializeListHead(&MiSegmentList);
ExInitializeFastMutex(&MiGlobalPageOperation);
KeInitializeEvent(&MmWaitPageEvent, SynchronizationEvent, FALSE);
// Until we're fully demand paged, we can do things the old way through
// the balance manager
MmInitializeMemoryConsumer(MC_CACHE, MiRosTrimCache);
KeInitializeGuardedMutex(&PsIdleProcess->AddressCreationLock);
MmKernelAddressSpace = &PsIdleProcess->Vm;

46
reactos/ntoskrnl/mm/rmap.c
View File

@ -10,9 +10,7 @@
/* INCLUDES *****************************************************************/
#include <ntoskrnl.h>
#ifdef NEWCC
#include "../cache/section/newmm.h"
#endif
#define NDEBUG
#include <debug.h>
@ -60,11 +58,23 @@ MmPageOutPhysicalAddress(PFN_NUMBER Page)
ExAcquireFastMutex(&RmapListLock);
entry = MmGetRmapListHeadPage(Page);
#ifdef NEWCC
// Special case for NEWCC: we can have a page that's only in a segment
// page table
if (entry && RMAP_IS_SEGMENT(entry->Address) && entry->Next == NULL)
return MmpPageOutPhysicalAddress(Page);
#endif
while (entry && RMAP_IS_SEGMENT(entry->Address))
entry = entry->Next;
if (entry == NULL)
{
ExReleaseFastMutex(&RmapListLock);
return(STATUS_UNSUCCESSFUL);
}
Process = entry->Process;
Address = entry->Address;
@ -112,8 +122,8 @@ MmPageOutPhysicalAddress(PFN_NUMBER Page)
Type = MemoryArea->Type;
if (Type == MEMORY_AREA_SECTION_VIEW)
{
Offset = (ULONG)((ULONG_PTR)Address - (ULONG_PTR)MemoryArea->StartingAddress
+ MemoryArea->Data.SectionData.ViewOffset);
Offset = (ULONG_PTR)Address - (ULONG_PTR)MemoryArea->StartingAddress
+ MemoryArea->Data.SectionData.ViewOffset.QuadPart;
/*
* Get or create a pageop
@ -143,6 +153,10 @@ MmPageOutPhysicalAddress(PFN_NUMBER Page)
Status = MmPageOutSectionView(AddressSpace, MemoryArea,
Address, PageOp);
}
else if (Type == MEMORY_AREA_CACHE)
{
Status = MmpPageOutPhysicalAddress(Page);
}
else if (Type == MEMORY_AREA_VIRTUAL_MEMORY)
{
PageOp = MmGetPageOp(MemoryArea, Address < MmSystemRangeStart ? Process->UniqueProcessId : NULL,
@ -197,9 +211,7 @@ MmSetCleanAllRmaps(PFN_NUMBER Page)
}
while (current_entry != NULL)
{
#ifdef NEWCC
if (!RMAP_IS_SEGMENT(current_entry->Address))
#endif
MmSetCleanPage(current_entry->Process, current_entry->Address);
current_entry = current_entry->Next;
}
@ -221,9 +233,7 @@ MmSetDirtyAllRmaps(PFN_NUMBER Page)
}
while (current_entry != NULL)
{
#ifdef NEWCC
if (!RMAP_IS_SEGMENT(current_entry->Address))
#endif
MmSetDirtyPage(current_entry->Process, current_entry->Address);
current_entry = current_entry->Next;
}
@ -246,9 +256,7 @@ MmIsDirtyPageRmap(PFN_NUMBER Page)
while (current_entry != NULL)
{
if (
#ifdef NEWCC
!RMAP_IS_SEGMENT(current_entry->Address) &&
#endif
MmIsDirtyPage(current_entry->Process, current_entry->Address))
{
ExReleaseFastMutex(&RmapListLock);
@ -268,9 +276,7 @@ MmInsertRmap(PFN_NUMBER Page, PEPROCESS Process,
PMM_RMAP_ENTRY current_entry;
PMM_RMAP_ENTRY new_entry;
ULONG PrevSize;
#ifdef NEWCC
if (!RMAP_IS_SEGMENT(Address))
#endif
Address = (PVOID)PAGE_ROUND_DOWN(Address);
new_entry = ExAllocateFromNPagedLookasideList(&RmapLookasideList);
@ -289,13 +295,12 @@ MmInsertRmap(PFN_NUMBER Page, PEPROCESS Process,
#endif
if (
#ifdef NEWCC
!RMAP_IS_SEGMENT(Address) &&
#endif
MmGetPfnForProcess(Process, Address) != Page)
{
DPRINT1("Insert rmap (%d, 0x%.8X) 0x%.8X which doesn't match physical "
"address 0x%.8X\n", Process->UniqueProcessId, Address,
"address 0x%.8X\n", Process ? Process->UniqueProcessId : 0,
Address,
MmGetPfnForProcess(Process, Address) << PAGE_SHIFT,
Page << PAGE_SHIFT);
KeBugCheck(MEMORY_MANAGEMENT);
@ -322,9 +327,7 @@ MmInsertRmap(PFN_NUMBER Page, PEPROCESS Process,
#endif
MmSetRmapListHeadPage(Page, new_entry);
ExReleaseFastMutex(&RmapListLock);
#ifdef NEWCC
if (!RMAP_IS_SEGMENT(Address))
#endif
{
if (Process == NULL)
{
@ -360,13 +363,12 @@ MmDeleteAllRmaps(PFN_NUMBER Page, PVOID Context,
}
MmSetRmapListHeadPage(Page, NULL);
ExReleaseFastMutex(&RmapListLock);
while (current_entry != NULL)
{
previous_entry = current_entry;
current_entry = current_entry->Next;
#ifdef NEWCC
if (!RMAP_IS_SEGMENT(previous_entry->Address))
#endif
{
if (DeleteMapping)
{
@ -384,12 +386,10 @@ MmDeleteAllRmaps(PFN_NUMBER Page, PVOID Context,
(void)InterlockedExchangeAddUL(&Process->Vm.WorkingSetSize, -PAGE_SIZE);
}
}
#ifdef NEWCC
else
{
ExFreeToNPagedLookasideList(&RmapLookasideList, previous_entry);
}
#endif
}
}
@ -419,9 +419,7 @@ MmDeleteRmap(PFN_NUMBER Page, PEPROCESS Process,
}
ExReleaseFastMutex(&RmapListLock);
ExFreeToNPagedLookasideList(&RmapLookasideList, current_entry);
#ifdef NEWCC
if (!RMAP_IS_SEGMENT(Address))
#endif
{
if (Process == NULL)
{
@ -440,7 +438,6 @@ MmDeleteRmap(PFN_NUMBER Page, PEPROCESS Process,
KeBugCheck(MEMORY_MANAGEMENT);
}
#ifdef NEWCC
PVOID
NTAPI
MmGetSegmentRmap(PFN_NUMBER Page, PULONG RawOffset)
@ -498,4 +495,3 @@ MmDeleteSectionAssociation(PFN_NUMBER Page)
}
ExReleaseFastMutex(&RmapListLock);
}
#endif

930
reactos/ntoskrnl/mm/section.c
View File

File diff suppressed because it is too large Load Diff

22
reactos/ntoskrnl/ntoskrnl-generic.rbuild
View File

@ -140,8 +140,8 @@
<file>view.c</file>
</directory>
</if>
<if property="NEWCC" value="1">
<directory name="cache">
<directory name="cache">
<if property="NEWCC" value="1">
<file>cachesub.c</file>
<file>copysup.c</file>
<file>fssup.c</file>
@ -149,16 +149,16 @@
<file>logsup.c</file>
<file>mdlsup.c</file>
<file>pinsup.c</file>
<directory name="section">
<file>data.c</file>
<file>fault.c</file>
<file>io.c</file>
<file>reqtools.c</file>
<file>sptab.c</file>
<file>swapout.c</file>
</directory>
</if>
<directory name="section">
<file>data.c</file>
<file>fault.c</file>
<file>io.c</file>
<file>reqtools.c</file>
<file>sptab.c</file>
<file>swapout.c</file>
</directory>
</if>
</directory>
<directory name="config">
<if property="ARCH" value="i386">
<directory name="i386">

11
reactos/ntoskrnl/po/poshtdwn.c
View File

@ -138,7 +138,7 @@ PopGracefulShutdown(IN PVOID Context)
/* Get the next process */
Process = PsGetNextProcess(Process);
}
/* First, the HAL handles any "end of boot" special functionality */
DPRINT1("HAL shutting down\n");
HalEndOfBoot();
@ -152,16 +152,17 @@ PopGracefulShutdown(IN PVOID Context)
CmShutdownSystem();
/* Note that modified pages should be written here (MiShutdownSystem) */
#ifdef NEWCC
/* Flush all user files before we start shutting down IO */
/* This is where modified pages are written back by the IO manager */
CcShutdownSystem();
#endif
/* In this step, the I/O manager does last-chance shutdown notification */
DPRINT1("I/O manager shutting down in phase 1\n");
IoShutdownSystem(1);
CcWaitForCurrentLazyWriterActivity();
#ifdef NEWCC
CcShutdownSystem();
#endif
/* Note that here, we should broadcast the power IRP to devices */
/* In this step, the HAL disables any wake timers */