-->
Definition
Examples
The following example creates a memory-mapped view of a part of an extremely large file and manipulates a portion of it.
Remarks
If you are using NIO connectors, Jetty uses memory-mapped files to do this. The problem is that on Windows, memory mapping a file causes the file to lock, so that you cannot update or replace the file. Effectively this means that you have to stop Jetty to update a file.
A memory-mapped file maps the contents of a file to an application's logical address space. Memory-mapped files enable programmers to work with extremely large files because memory can be managed concurrently, and they allow complete, random access to a file without the need for seeking. Memory-mapped files can also be shared across multiple processes.
The CreateFromFile methods create a memory-mapped file from a specified path or a FileStream of an existing file on disk. Changes are automatically propagated to disk when the file is unmapped.
The CreateNew methods create a memory-mapped file that is not mapped to an existing file on disk; and are suitable for creating shared memory for interprocess communication (IPC).
A memory-mapped file can be associated with an optional name that allows the memory-mapped file to be shared with other processes.
You can create multiple views of the memory-mapped file, including views of parts of the file. You can map the same part of a file to more than one address to create concurrent memory. For two views to remain concurrent, they have to be created from the same memory-mapped file. Creating two file mappings of the same file with two views does not provide concurrency.
Properties
| SafeMemoryMappedFileHandleSafeMemoryMappedFileHandleSafeMemoryMappedFileHandleSafeMemoryMappedFileHandle | Gets the file handle of a memory-mapped file. |
Methods
| CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, HandleInheritability, Boolean)CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, HandleInheritability, Boolean)CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, HandleInheritability, Boolean)CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, HandleInheritability, Boolean) | Creates a memory-mapped file from an existing file with the specified access mode, name, inheritability, and capacity. |
| CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, MemoryMappedFileSecurity, HandleInheritability, Boolean)CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, MemoryMappedFileSecurity, HandleInheritability, Boolean)CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, MemoryMappedFileSecurity, HandleInheritability, Boolean)CreateFromFile(FileStream, String, Int64, MemoryMappedFileAccess, MemoryMappedFileSecurity, HandleInheritability, Boolean) | Creates a memory-mapped file that has the specified name, capacity, access type, security permissions, inheritability, and disposal requirement from a file on disk. |
| CreateFromFile(String)CreateFromFile(String)CreateFromFile(String)CreateFromFile(String) | Creates a memory-mapped file from a file on disk. |
| CreateFromFile(String, FileMode)CreateFromFile(String, FileMode)CreateFromFile(String, FileMode)CreateFromFile(String, FileMode) | Creates a memory-mapped file that has the specified access mode from a file on disk. |
| CreateFromFile(String, FileMode, String)CreateFromFile(String, FileMode, String)CreateFromFile(String, FileMode, String)CreateFromFile(String, FileMode, String) | Creates a memory-mapped file that has the specified access mode and name from a file on disk. |
| CreateFromFile(String, FileMode, String, Int64)CreateFromFile(String, FileMode, String, Int64)CreateFromFile(String, FileMode, String, Int64)CreateFromFile(String, FileMode, String, Int64) | Creates a memory-mapped file that has the specified access mode, name, and capacity from a file on disk. |
| CreateFromFile(String, FileMode, String, Int64, MemoryMappedFileAccess)CreateFromFile(String, FileMode, String, Int64, MemoryMappedFileAccess)CreateFromFile(String, FileMode, String, Int64, MemoryMappedFileAccess)CreateFromFile(String, FileMode, String, Int64, MemoryMappedFileAccess) | Creates a memory-mapped file that has the specified access mode, name, capacity, and access type from a file on disk. |
| CreateNew(String, Int64)CreateNew(String, Int64)CreateNew(String, Int64)CreateNew(String, Int64) | Creates a memory-mapped file that has the specified capacity in system memory. |
| CreateNew(String, Int64, MemoryMappedFileAccess)CreateNew(String, Int64, MemoryMappedFileAccess)CreateNew(String, Int64, MemoryMappedFileAccess)CreateNew(String, Int64, MemoryMappedFileAccess) | Creates a memory-mapped file that has the specified capacity and access type in system memory. |
| CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability)CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability)CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability)CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability) | Creates a memory-mapped file that has the specified name, capacity, access type, memory allocation options and inheritability. |
| CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability)CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability)CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability)CreateNew(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability) | Creates a memory-mapped file that has the specified capacity, access type, memory allocation, security permissions, and inheritability in system memory. |
| CreateOrOpen(String, Int64)CreateOrOpen(String, Int64)CreateOrOpen(String, Int64)CreateOrOpen(String, Int64) | Creates or opens a memory-mapped file that has the specified name and capacity in system memory. |
| CreateOrOpen(String, Int64, MemoryMappedFileAccess)CreateOrOpen(String, Int64, MemoryMappedFileAccess)CreateOrOpen(String, Int64, MemoryMappedFileAccess)CreateOrOpen(String, Int64, MemoryMappedFileAccess) | Creates or opens a memory-mapped file that has the specified name, capacity and access type in system memory. |
| CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability)CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability)CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability)CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, HandleInheritability) | Creates a new empty memory mapped file or opens an existing memory mapped file if one exists with the same name. If opening an existing file, the capacity, options, and memory arguments will be ignored. |
| CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability)CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability)CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability)CreateOrOpen(String, Int64, MemoryMappedFileAccess, MemoryMappedFileOptions, MemoryMappedFileSecurity, HandleInheritability) | Creates or opens a memory-mapped file that has the specified name, capacity, access type, memory allocation, security permissions, and inheritability in system memory. |
| CreateViewAccessor()CreateViewAccessor()CreateViewAccessor()CreateViewAccessor() | Creates a MemoryMappedViewAccessor that maps to a view of the memory-mapped file. |
| CreateViewAccessor(Int64, Int64)CreateViewAccessor(Int64, Int64)CreateViewAccessor(Int64, Int64)CreateViewAccessor(Int64, Int64) | Creates a MemoryMappedViewAccessor that maps to a view of the memory-mapped file, and that has the specified offset and size. |
| CreateViewAccessor(Int64, Int64, MemoryMappedFileAccess)CreateViewAccessor(Int64, Int64, MemoryMappedFileAccess)CreateViewAccessor(Int64, Int64, MemoryMappedFileAccess)CreateViewAccessor(Int64, Int64, MemoryMappedFileAccess) | Creates a MemoryMappedViewAccessor that maps to a view of the memory-mapped file, and that has the specified offset, size, and access restrictions. |
| CreateViewStream()CreateViewStream()CreateViewStream()CreateViewStream() | Creates a stream that maps to a view of the memory-mapped file. |
| CreateViewStream(Int64, Int64)CreateViewStream(Int64, Int64)CreateViewStream(Int64, Int64)CreateViewStream(Int64, Int64) | Creates a stream that maps to a view of the memory-mapped file, and that has the specified offset and size. |
| CreateViewStream(Int64, Int64, MemoryMappedFileAccess)CreateViewStream(Int64, Int64, MemoryMappedFileAccess)CreateViewStream(Int64, Int64, MemoryMappedFileAccess)CreateViewStream(Int64, Int64, MemoryMappedFileAccess) | Creates a stream that maps to a view of the memory-mapped file, and that has the specified offset, size, and access type. |
| Dispose()Dispose()Dispose()Dispose() | Releases all resources used by the MemoryMappedFile. |
| Dispose(Boolean)Dispose(Boolean)Dispose(Boolean)Dispose(Boolean) | Releases the unmanaged resources used by the MemoryMappedFile and optionally releases the managed resources. |
| Equals(Object)Equals(Object)Equals(Object)Equals(Object) | Determines whether the specified object is equal to the current object. (Inherited from Object) |
| GetAccessControl()GetAccessControl()GetAccessControl()GetAccessControl() | Gets the access control to the memory-mapped file resource. |
| GetHashCode()GetHashCode()GetHashCode()GetHashCode() | Serves as the default hash function. (Inherited from Object) |
| GetType()GetType()GetType()GetType() | Gets the Type of the current instance. (Inherited from Object) |
| MemberwiseClone()MemberwiseClone()MemberwiseClone()MemberwiseClone() | Creates a shallow copy of the current Object. (Inherited from Object) |
| OpenExisting(String)OpenExisting(String)OpenExisting(String)OpenExisting(String) | Opens an existing memory-mapped file that has the specified name in system memory. |
| OpenExisting(String, MemoryMappedFileRights)OpenExisting(String, MemoryMappedFileRights)OpenExisting(String, MemoryMappedFileRights)OpenExisting(String, MemoryMappedFileRights) | Opens an existing memory-mapped file that has the specified name and access rights in system memory. |
| OpenExisting(String, MemoryMappedFileRights, HandleInheritability)OpenExisting(String, MemoryMappedFileRights, HandleInheritability)OpenExisting(String, MemoryMappedFileRights, HandleInheritability)OpenExisting(String, MemoryMappedFileRights, HandleInheritability) | Opens an existing memory-mapped file that has the specified name, access rights, and inheritability in system memory. |
| SetAccessControl(MemoryMappedFileSecurity)SetAccessControl(MemoryMappedFileSecurity)SetAccessControl(MemoryMappedFileSecurity)SetAccessControl(MemoryMappedFileSecurity) | Sets the access control to the memory-mapped file resource. |
| ToString()ToString()ToString()ToString() | Returns a string that represents the current object. (Inherited from Object) |
Applies to
See also
I have 2 threads that concurrently access the same large file(.txt).
1st Thread is reading from the File.2nd Thread is writing to the File.
Both threads access the same block e.g. (start:0, blocksize:10), but with different channel & Buffer instances
Reader:
Writer:
I know that if both starts at the same time, I will get Overlapping Exceptions! BUT what I would like to know, at which point exactly the Overlapping is happing? I mean when occurs the 'lock' exactly?Example: lets say the writer get access first, then if reader try to access, at which point is it possible?:
1, 2, 3 or 4?
No 4 is sure, because the channel is been closed! What about the other points?
Thanks!
Steve C
Rami.QMemory Mapped File
2 Answers
Memory Mapped File Python
I am summing up a few notes from a chat conversation with the OP. The OP had the mental model (like most of us) that once a thread writes to a data structure, that data structure is immediately visible to all other threads. In the OPs tests using memory mapped files, he had confirmed that this appeared to be true on a single socket Intel CPU.
Unfortunately this is not true, and is an area where Java can and does show the underlying behaviour of the hardware. Java has been designed to assume that code is single threaded, and can thus be optimised as such until such times as it is told otherwise. What that means will differ by hardware, and version of hotspot (and the statistics that hotspot has collected). This complexity, and running on a single socket Intel CPU invalidated the OPs test.
For further information, the following links will help gain a deeper understanding into the 'Java Memory Model'. And particularly that synchronized does not just mean 'mutual exclusion'; in hardware terms it is also about 'data visibility' and 'instruction ordering'. Two topics that single threaded code take for granted.
Do not worry if this takes time to sink in, and that you feel overwhelmed at first. We all felt like that at first. Java does an amazing job of hiding this complexity, if and only if you follow this one simple rule. When a thread reads or modifies a shared data structure, it must be within a synchronized block. That is, both the writing thread and the reading thread. Obviously I am simplifying, but follow that rule and the program will always work. Break it only if you have a very deep understanding of the Java Memory Model, memory barriers and how it relates to different hardware (and even then concurrency experts even avoid breaking that rule too if they can; going single threaded is often much much simpler and can be surprisingly fast.. many low latency systems are designed to be mostly single threaded for this reason).
To directly answer the OPs question. The sample code from the question has no locks in it. No memory barriers, no concurrency controls at all. Thus the behaviour of how the reads and writes will interact is undefined. They may work, they may not. They may work most of the time. Intel has the strongest memory guarantees of all CPUs, and running the test cases on a single socket Intel CPU would miss a lot of complex bugs. Sun was caught out by this too before Java 5 and JSR 133 came out (read the article on why Double Checked Locking was broken in Java for more detail).
Chris KChris KFile Locking Software
You won't get any locking exceptions from this code, or any blocks either. File locks operate between processes, not between threads. What you need here is synchronization, or semaphores, or ReadWriteLocks. And there's no need to use two channels.
user207421user207421