Commit Graph

104 Commits

Author SHA1 Message Date
Serge Gautherie
c2e454b439 [NTOS:CC] CcRosFlushVacb(): Fix Iosb annotation
Addendum to 2ba1926.
2021-06-22 10:33:45 +02:00
Timo Kreuzer
dd08ae2c0f [NTOS:CC] Fix use of unintialized variable (caught by RTC1) 2021-05-24 22:00:11 +02:00
Jérôme Gardou
0187c1e113 [NTOS:MM] Fix PFN tracing 2021-03-30 16:26:43 +02:00
Jérôme Gardou
2a962eaf8c [NTOS:CC] Keep a reference on the shared cache map of the file when we are in lazy write
This should fix "Leaking VACB" debug prints
2021-02-19 15:48:31 +01:00
Jérôme Gardou
cc9607e94e [NTOS:CC] Fix use of uninitialized variable 2021-02-03 13:35:17 +01:00
Jérôme Gardou
b7eb0fddf3 Address PR review 2021-02-03 09:41:24 +01:00
Jérôme Gardou
2ba1926037 [NTOS:MM][NTOS:CC] Performance improvement again
Read files by 64kb chunks instead of page-sized chunks.
2021-02-03 09:41:23 +01:00
Jérôme Gardou
d0bf98663b [NTOS:CC] Be sure to flush the whole file in CcFlushCache 2021-02-03 09:41:23 +01:00
Jérôme Gardou
5949d5095d [NTOS:CC][NTOS:MM] Try respecting ValidDataLength 2021-02-03 09:41:23 +01:00
Jérôme Gardou
bdb73edab7 [NTOS:CC] Flush the whole VACB
Let Mm know what it has to do.
2021-02-03 09:41:23 +01:00
Jérôme Gardou
804f5a41ed [NTOS:CC] Improve trace messages 2021-02-03 09:41:23 +01:00
Jérôme Gardou
20fe42c9e9 [NTOS:CC] Simplify CcFlushCache implementation 2021-02-03 09:41:23 +01:00
Jérôme Gardou
9b6240ee03 [NTOS:CC] Get rid of ROS_VACB:Valid 2021-02-03 09:41:23 +01:00
Jérôme Gardou
33cde28312 [NTOS:CC] Simplify CcRosDeleteFileCache 2021-02-03 09:41:23 +01:00
Jérôme Gardou
a9193b5cc2 [NTOS:CC] Remove dead code 2021-02-03 09:41:23 +01:00
Jérôme Gardou
57ee31ee33 [NTOS:CC] Perform sanity checks before doing anything else 2021-02-03 09:41:23 +01:00
Jérôme Gardou
8287a098b9 [NTOS:CC] Fix potnetial use-after-free 2021-02-03 09:41:23 +01:00
Jérôme Gardou
1505abbc09 [NTOS:CC] Do not write behind concurrently the same file 2021-02-03 09:41:23 +01:00
Jérôme Gardou
347a4f146b [NTOS] Loop again and again until the whole cache is empty when sutting down 2021-02-03 09:41:23 +01:00
Jérôme Gardou
6d97d8d2e1 [NTOS:CC] Fix some tests, complain where the current implementation won't let us do the right thing 2021-02-03 09:41:22 +01:00
Jérôme Gardou
30f71c7fc0 [NTOS] Zero data unconditionally after segment end, unless section is created with SEC_RESERVE
Use a SEC_RESERVE section in Cc
2021-02-03 09:41:22 +01:00
Jérôme Gardou
f8aa14ce4e [NTOS:CC] Acquire file for flush when flushing if necessary 2021-02-03 09:41:22 +01:00
Jérôme Gardou
70c62aa2c9 [NTOS:CC] Fix Vacb size usage & check 2021-02-03 09:41:22 +01:00
Jérôme Gardou
c74cbf0c0b [NTOS/CC] Be more precise when notifying Mm about dirty pages 2021-02-03 09:41:22 +01:00
Jérôme Gardou
36e18aab35 [NTOS:CC] Remove unused functions 2021-02-03 09:41:22 +01:00
Jérôme Gardou
d8cdb89fb0 [NTOSKRNL] Overhaul Cc and Mm relationship
Previously, when creating a file section, Mm requested Cc to cache the file, then Cc would request pages from Mm, then Mm would request them back to serve its file-mapping role
Now, Mm does it all by itself. If file cahcing is requested by the FS driver, then Cc creates a file mapping and uses that to serve its purpose.

This is a rewrite of Cc
2021-02-03 09:41:22 +01:00
Jérôme Gardou
8631e75837 [NTOS:CC] Acquire the master lock after freeing the VACB in CcRosFlushDirtyPages
Fixes a random ASSERT
2020-12-09 18:06:42 +01:00
Victor Perevertkin
5c7ce4475e
[REACTOS] Cleanup INIT and some PAGE section allocations
- Change INIT_FUNCTION and INIT_SECTION to CODE_SEG("INIT") and DATA_SEG("INIT") respectively
- Remove INIT_FUNCTION from function prototypes
- Remove alloc_text pragma calls as they are not needed anymore
2020-11-02 21:45:31 +03:00
Jérôme Gardou
1c528cbf84 Revert "[NTOS/MM]
- Fix PFNs tracing
     - Add private pages to the process working set"

This reverts commit 4c5351bf55.
Not ready for prime time
2020-10-20 15:56:21 +02:00
Jérôme Gardou
4c5351bf55 [NTOS/MM]
- Fix PFNs tracing
 - Add private pages to the process working set
2020-10-20 15:20:59 +02:00
Serge Gautherie
b20f815126
[NTOSKRNL] Place INIT_FUNCTION before the return type (#2823)
(but after 'static' or SAL 2 annotation.)
Follow-up to 71fefa32, which mentions that it's actually required by the compiler in some circumstances.
2020-05-23 15:56:10 +02:00
Pierre Schweitzer
77b6899d89
[NTOSKRNL] Don't set VACB dirty on release if already dirty
CORE-15954
2019-04-20 11:23:35 +02:00
Pierre Schweitzer
1a267045f8
[NTOSKRNL] Honor files that shouldn't be lazy written 2018-12-23 12:10:58 +01:00
Pierre Schweitzer
1435ff95b4
[NTOSKRNL] Don't call AcquireForLazyWrite with the master lock held
This incorrect behavior was leading to a call at too high IRQL for paged code.
This was triggered by MS FastFAT.

ReleaseFromLazyWrite call was already correctly called to that regard.

CORE-11819
2018-12-21 08:46:40 +01:00
Pierre Schweitzer
4f8b041bf0
[NTOSKRNL] Drop the ViewLock mutex in favour of the master spin lock
This will allow Cc calls during DPC, which is required by MS FastFAT

CORE-11819
2018-12-19 22:51:45 +01:00
Pierre Schweitzer
182cc5c5ab
[NTOSKRNL] Don't dereference VACB when allocating its memory area fails
This avoids performing a double-free (even though that's hidden by the
fact we use lookaside allocations for VACB), and it avoids freeing
a memory address at an uninitialized address.
We don't care about references here, the VACB was just allocated, never
linked and we're its only user.

CORE-15413
2018-12-08 19:56:03 +01:00
Pierre Schweitzer
f284947622
[NTOSKRNL] Move the PinCount out of the VACB to the BCB
Given current ReactOS implementation, a VACB can be pinned
several times, with different BCB. In next commits, a single
BCB will be able to be pinned several times. That would
lead to severe inconsistencies in counting and thus corruption.
2018-10-05 21:26:16 +02:00
Pierre Schweitzer
e17f61138c
[NTOSKRNL] When allocating a new BCB, save it in a list
This list is stored in the shared map. Later, this will allow
reusing BCB when appropriate
2018-09-05 22:06:25 +02:00
Thomas Faber
1d398057a3
[NTOS:CC] Access SectionObjectPointers without lock in CcRosInitializeFileCache. CORE-14691
kmtest:NtCreateSection calls CcInitializeCacheMap with a
NULL value for SectionObjectPointers. This will cause an exception when
trying to access it, which in Windows can be handled gracefully.
However accessing it while holding ViewLock means the lock will not be
released, leading to an APC_INDEX_MISMATCH bugcheck.

This solves the problem by allocating SharedCacheMap outside the lock,
then freeing it again under lock if another thread has updated SharedCacheMap
in the mean time. This is also What Windows Does(TM).
2018-06-05 16:24:13 +02:00
Pierre Schweitzer
2cf9a69bce
[NTOSKRNL] Addendum to 8a8cb4d: don't print uninit pointer. 2018-05-23 08:44:43 +02:00
Pierre Schweitzer
8a8cb4d890
[NTOSKRNL] Only consider SharedCacheMap value once ViewLock is acquired.
This avoids a really nasty race condition in our cache controler where
two concurrents could try to initialize cache on the same file.
This had two nasty effects: first shared map was purely leaked and erased
by the second one. And the private cache map, allocated on the first shared
cache map couldn't be freed and was leading to Mm BSOD (free in a middle of
a block).

This was often triggered while building ReactOS on ReactOS (with multi threads).
With that patch, I cannot crash anylonger while building ReactOS.

CORE-14634
2018-05-23 08:41:46 +02:00
Pierre Schweitzer
54c049bd6e
[NTOKSNRL] Always flush dirty VACB.
Recent changes seem to show that it's not
required to be exclusive on VACB to be able
to flush it.

This commit goes with f2c44aa and fixes the
last issues going with copying huge files.
There are no longer BSODs (be it in Mm or Cc).
I could, with 750MB RAM extract a 2GB file from
a 53MB archive and copy a 2,5GB file from a VBox
share to the disk. Note that writes are often
deferred, so if copy works, it's not that fast for now.

Note that it also brings some beloved behavior from
Windows: copy times are totally unreliable now when
writes are deferred. Little remaining times when
actively copying, high remaining times when deferred
writes in action. And goes between both... Sorry! ;-)

https://xkcd.com/612/

CORE-9696
CORE-11175
2018-04-30 22:24:30 +02:00
Pierre Schweitzer
74c5d8b6bd
[NTOSKRNL] Free unused VACB when required.
Same mechanism exists in Windows (even their Cc
is way different from ours...) where when Cc is
out of memory (in their case, out of VACB), we
will start scavenge old & unused VACB to free
some of the memory.

It's useful in case we're operating we big files
operations, we may run out of memory where to map
VACB for them, so start to scavenge VACB to free
some of that memory.

With this, I am able to install Qt 4.8.6 with 2,5GB of RAM,
scavenging acting when needed!

CORE-12081
CORE-14582
2018-04-30 12:10:24 +02:00
Pierre Schweitzer
cc54e51495
[NTOSKRNL] Unmark dirty first, and then write.
This will avoid trying to flush twice a dirty VACB under
high IOs pressure.

CORE-14584
2018-04-30 10:36:19 +02:00
Pierre Schweitzer
f2c44aa483
[NTOSKRNL] Fix lazy writer for in-use VACB.
Adjusting refcount and enabling lazy-write for pinned
VACB makes it actually more efficient, often purging
data to disk, reducing memory stress for the system.

This is required for defering writes.

This commit unfortunately (?) reverts a previous revert.

CORE-12081
CORE-14582
CORE-14313
2018-04-29 20:42:53 +02:00
Pierre Schweitzer
2ea6de8a42
[NTOSKRNL] Also try to extract name from FCB when leaking VACB 2018-04-27 19:01:35 +02:00
Pierre Schweitzer
43836b0fbb
[NTOSKRNL] In !filecache, try to display FCB name
When no name is set in the file object, try to read the name
from the FCB. We only support FastFAT (ours) FCB for now.

This is clearly a hack, but for a kdbg command, so ;-)
2018-04-27 18:57:30 +02:00
Pierre Schweitzer
579a784e04
[NTOSKNRL] In case we leak a VACB, debug as much information as possible.
CORE-14578
2018-04-27 14:14:56 +02:00
Pierre Schweitzer
fcf83315dc
[NTOSKRNL] Noisily dereference mapped VACB on cache release.
It seems that on process killing, some VACB may be deleted while
still mapped. With current reference counting, they will actually
not be deleted, but leaked, and an ASSERT will be triggered.

CORE-14578
2018-04-27 10:23:06 +02:00
Pierre Schweitzer
fd3a6c1089 [NTOSKRNL] Properly reset VACB on free
CID 1434271
2018-04-15 22:52:53 +02:00