Struct faiss::IndexShardsTemplate

template<typename IndexT>
struct faiss::IndexShardsTemplate : public ThreadedIndex<IndexT>

Index that concatenates the results from several sub-indexes

Public Types

using idx_t = typename IndexT::idx_t
using component_t = typename IndexT::component_t
using distance_t = typename IndexT::distance_t

Public Functions

explicit IndexShardsTemplate(bool threaded = false, bool successive_ids = true)

The dimension that all sub-indices must share will be the dimension of the first sub-index added

Parameters

  • threaded – do we use one thread per sub_index or do queries sequentially?

  • successive_ids – should we shift the returned ids by the size of each sub-index or return them as they are?

explicit IndexShardsTemplate(idx_t d, bool threaded = false, bool successive_ids = true)
Parameters

  • threaded – do we use one thread per sub_index or do queries sequentially?

  • successive_ids – should we shift the returned ids by the size of each sub-index or return them as they are?

explicit IndexShardsTemplate(int d, bool threaded = false, bool successive_ids = true)

int version due to the implicit bool conversion ambiguity of int as dimension

inline void add_shard(IndexT *index)

Alias for addIndex()

inline void remove_shard(IndexT *index)

Alias for removeIndex()

void add(idx_t n, const component_t *x) override

supported only for sub-indices that implement add_with_ids

void add_with_ids(idx_t n, const component_t *x, const idx_t *xids) override

Cases (successive_ids, xids):

  • true, non-NULL ERROR: it makes no sense to pass in ids and request them to be shifted

  • true, NULL OK, but should be called only once (calls add() on sub-indexes).

  • false, non-NULL OK: will call add_with_ids with passed in xids distributed evenly over shards

  • false, NULL OK: will call add_with_ids on each sub-index, starting at ntotal

void search(idx_t n, const component_t *x, idx_t k, distance_t *distances, idx_t *labels) const override
void train(idx_t n, const component_t *x) override
void syncWithSubIndexes()

Synchronize the top-level index (IndexShards) with data in the sub-indices

Public Members

bool successive_ids

Protected Functions

void onAfterAddIndex(IndexT *index) override

Called just after an index is added.

void onAfterRemoveIndex(IndexT *index) override

Called just after an index is removed.