Namespace faiss::ivflib

namespace ivflib

Functions

void check_compatible_for_merge(const Index *index1, const Index *index2)

check if two indexes have the same parameters and are trained in the same way, otherwise throw.

const IndexIVF *extract_index_ivf(const Index *index)

get an IndexIVF from an index. The index may be an IndexIVF or some wrapper class that encloses an IndexIVF

throws an exception if this is not the case.

IndexIVF *extract_index_ivf(Index *index)
const IndexIVF *try_extract_index_ivf(const Index *index)

same as above but returns nullptr instead of throwing on failure

IndexIVF *try_extract_index_ivf(Index *index)
void merge_into(Index *index0, Index *index1, bool shift_ids)

Merge index1 into index0. Works on IndexIVF’s and IndexIVF’s embedded in a IndexPreTransform. On output, the index1 is empty.

Parameters:

shift_ids – translate the ids from index1 to index0->prev_ntotal

void search_centroid(Index *index, const float *x, int n, idx_t *centroid_ids)
void search_and_return_centroids(Index *index, size_t n, const float *xin, long k, float *distances, idx_t *labels, idx_t *query_centroid_ids, idx_t *result_centroid_ids)
ArrayInvertedLists *get_invlist_range(const Index *index, long i0, long i1)

Get a subset of inverted lists [i0, i1)

void set_invlist_range(Index *index, long i0, long i1, ArrayInvertedLists *src)

Set a subset of inverted lists.

void search_with_parameters(const Index *index, idx_t n, const float *x, idx_t k, float *distances, idx_t *labels, const IVFSearchParameters *params, size_t *nb_dis = nullptr, double *ms_per_stage = nullptr)

search an IndexIVF, possibly embedded in an IndexPreTransform with given parameters. This is a way to set the nprobe and get statdistics in a thread-safe way.

Optionally returns (if non-nullptr):

  • nb_dis: number of distances computed

  • ms_per_stage: [0]: preprocessing time [1]: coarse quantization, [2]: list scanning

void range_search_with_parameters(const Index *index, idx_t n, const float *x, float radius, RangeSearchResult *result, const IVFSearchParameters *params, size_t *nb_dis = nullptr, double *ms_per_stage = nullptr)

same as search_with_parameters but for range search

IndexIVFResidualQuantizer *ivf_residual_from_quantizer(const ResidualQuantizer&, int nlevel)

Build an IndexIVFResidualQuantizer from an ResidualQuantizer, using the nlevel first components as coarse quantizer and the rest as codes in invlists

void ivf_residual_add_from_flat_codes(IndexIVFResidualQuantizer *ivfrq, size_t ncode, const uint8_t *codes, int64_t code_size = -1)

add from codes. NB that the norm component is not used, so the code_size can be provided.

Parameters:

  • ivfrq – index to populate with the codes

  • codes – codes to add, size (ncode, code_size)

  • code_size – override the ivfrq’s code_size, useful if the norm encoding is different

struct SlidingIndexWindow
#include <IVFlib.h>

A set of IndexIVFs concatenated together in a FIFO fashion. at each “step”, the oldest index slice is removed and a new index is added.

Public Functions

SlidingIndexWindow(Index *index)

index should be initially empty and trained

void step(const Index *sub_index, bool remove_oldest)

Add one index to the current index and remove the oldest one.

Parameters:

  • sub_index – slice to swap in (can be NULL)

  • remove_oldest – if true, remove the oldest slices

Public Members

Index *index

common index that contains the sliding window

ArrayInvertedLists *ils

InvertedLists of index.

int n_slice

number of slices currently in index

size_t nlist

same as index->nlist

std::vector<std::vector<size_t>> sizes

cumulative list sizes at each slice