public class MemoryManagerImpl extends Object implements MemoryManager
This class is not thread safe, but is re-entrant - ensure creation and all invocations are triggered from the same thread.
MemoryManager.Callback
Constructor and Description |
---|
MemoryManagerImpl(Configuration conf)
Create the memory manager.
|
MemoryManagerImpl(long poolSize)
Create the memory manager
|
Modifier and Type | Method and Description |
---|---|
void |
addedRow(int rows)
Give the memory manager an opportunity for doing a memory check.
|
void |
addWriter(Path path,
long requestedAllocation,
MemoryManager.Callback callback)
Add a new writer's memory allocation to the pool.
|
long |
checkMemory(long previous,
MemoryManager.Callback writer)
As part of adding rows, the writer calls this method to determine
if the scale factor has changed.
|
double |
getAllocationScale()
The scaling factor for each allocation to ensure that the pool isn't
oversubscribed.
|
long |
getTotalMemoryPool()
Get the total pool size that is available for ORC writers.
|
void |
notifyWriters()
Deprecated.
remove this method
|
void |
removeWriter(Path path)
Remove the given writer from the pool.
|
public MemoryManagerImpl(Configuration conf)
conf
- use the configuration to find the maximum size of the memory
pool.public MemoryManagerImpl(long poolSize)
poolSize
- the size of memory to usepublic void addWriter(Path path, long requestedAllocation, MemoryManager.Callback callback) throws IOException
addWriter
in interface MemoryManager
path
- the file that is being writtenrequestedAllocation
- the requested buffer sizeIOException
public void removeWriter(Path path) throws IOException
removeWriter
in interface MemoryManager
path
- the file that has been closedIOException
public long getTotalMemoryPool()
public double getAllocationScale()
public void addedRow(int rows) throws IOException
MemoryManager
addedRow
in interface MemoryManager
rows
- number of rows addedIOException
public void notifyWriters() throws IOException
IOException
public long checkMemory(long previous, MemoryManager.Callback writer) throws IOException
MemoryManager
checkMemory
in interface MemoryManager
previous
- the previous allocationwriter
- the callback to call back into if we need toIOException
Copyright © 2013–2023 The Apache Software Foundation. All rights reserved.