Class faiss::gpu::StackDeviceMemory¶

class StackDeviceMemory¶

Device memory manager that provides temporary memory allocations out of a region of memory, for a single device

Public Functions

StackDeviceMemory(GpuResources *res, int device, size_t allocPerDevice)¶: Allocate a new region of memory that we manage.

StackDeviceMemory(int device, void *p, size_t size, bool isOwner)¶: Manage a region of memory for a particular device, with or without ownership

~StackDeviceMemory()¶

int getDevice() const¶

void *allocMemory(cudaStream_t stream, size_t size)¶: All allocations requested should be a multiple of 16 bytes.

void deallocMemory(int device, cudaStream_t, size_t size, void *p)¶

size_t getSizeAvailable() const¶

std::string toString() const¶

Protected Attributes

int device_¶: Our device.

Stack stack_¶: Memory stack.

struct Range¶

Previous allocation ranges and the streams for which synchronization is required

Public Functions

inline Range(char *s, char *e, cudaStream_t str)¶

Public Members

char *start_¶

char *end_¶

cudaStream_t stream_¶

struct Stack¶

Public Functions

Stack(GpuResources *res, int device, size_t size)¶: Constructor that allocates memory via cudaMalloc.

~Stack()¶

size_t getSizeAvailable() const¶: Returns how much size is available for an allocation without calling cudaMalloc

char *getAlloc(size_t size, cudaStream_t stream)¶: Obtains an allocation; all allocations are guaranteed to be 16 byte aligned

void returnAlloc(char *p, size_t size, cudaStream_t stream)¶: Returns an allocation.

std::string toString() const¶: Returns the stack state.

Public Members

GpuResources *res_¶: Our GpuResources object.

int device_¶: Device this allocation is on.

char *alloc_¶: Where our temporary memory buffer is allocated; we allocate starting 16 bytes into this

size_t allocSize_¶: Total size of our allocation.

char *start_¶: Our temporary memory region; [start_, end_) is valid.

char *end_¶

char *head_¶: Stack head within [start, end)

std::list<Range> lastUsers_¶: List of previous last users of allocations on our stack, for possible synchronization purposes

size_t highWaterMemoryUsed_¶: What’s the high water mark in terms of memory used from the temporary buffer?