Faiss

- Main Page
- Namespaces
- Classes
- Files

- Class List
- Class Index
- Class Hierarchy
- Class Members

All Classes Namespaces Functions Variables Typedefs Enumerations Enumerator Friends
Public Member Functions | Public Attributes | List of all members
faiss::IndexBinaryIVF Struct Reference

`#include <IndexBinaryIVF.h>`

Inheritance diagram for faiss::IndexBinaryIVF:

## Public Member Functions
void | train_q1 (size_t n, const uint8_t *x, bool verbose)
| Trains the quantizer and calls train_residual to train sub-quantizers.
| IndexBinaryIVF (IndexBinary *quantizer, size_t d, size_t nlist)
void | reset () override
| Removes all elements from the database.
void | train (idx_t n, const uint8_t *x) override
| Trains the quantizer and calls train_residual to train sub-quantizers.
void | add (idx_t n, const uint8_t *x) override
| Quantizes x and calls add_with_key.
void | add_with_ids (idx_t n, const uint8_t *x, const long *xids) override
void | add_core (idx_t n, const uint8_t *x, const long *xids, const long *precomputed_idx)
| same as add_with_ids, with precomputed coarse quantizer
void | search_preassigned (idx_t n, const uint8_t *x, idx_t k, const idx_t *assign, const int32_t *centroid_dis, int32_t *distances, idx_t *labels, bool store_pairs, const IVFSearchParameters *params=nullptr) const
virtual BinaryInvertedListScanner * | get_InvertedListScanner (bool store_pairs=false) const
virtual void | search (idx_t n, const uint8_t *x, idx_t k, int32_t *distances, idx_t *labels) const override
void | reconstruct (idx_t key, uint8_t *recons) const override
void | reconstruct_n (idx_t i0, idx_t ni, uint8_t *recons) const override
void | search_and_reconstruct (idx_t n, const uint8_t *x, idx_t k, int32_t *distances, idx_t *labels, uint8_t *recons) const override
virtual void | reconstruct_from_offset (long list_no, long offset, uint8_t *recons) const
long | remove_ids (const IDSelector &sel) override
| Dataset manipulation functions.
virtual void | merge_from (IndexBinaryIVF &other, idx_t add_id)
size_t | get_list_size (size_t list_no) const
void | make_direct_map (bool new_maintain_direct_map=true)
double | imbalance_factor () const
| 1= perfectly balanced, >1: imbalanced
void | print_stats () const
| display some stats about the inverted lists
void | replace_invlists (InvertedLists *il, bool own=false)
Public Member Functions inherited from faiss::IndexBinary
| IndexBinary (idx_t d=0, MetricType metric=METRIC_L2)
virtual void | range_search (idx_t n, const uint8_t *x, int radius, RangeSearchResult *result) const
void | assign (idx_t n, const uint8_t *x, idx_t *labels, idx_t k=1)
void | display () const

## Public Attributes
InvertedLists * | invlists
| Acess to the actual data.
bool | own_invlists
size_t | nprobe
| number of probes at query time
size_t | max_codes
| max nb of codes to visit to do a query
bool | use_heap = true
bool | maintain_direct_map
| map for direct access to the elements. Enables reconstruct().
std::vector< long > | direct_map
IndexBinary * | quantizer
| quantizer that maps vectors to inverted lists
size_t | nlist
| number of possible key values
bool | own_fields
| whether object owns the quantizer
ClusteringParameters | cp
| to override default clustering params
Index * | clustering_index
| to override index used during clustering
Public Attributes inherited from faiss::IndexBinary
int | d
| vector dimension
int | code_size
| number of bytes per vector ( = d / 8 )
idx_t | ntotal
| total nb of indexed vectors
bool | verbose
| verbosity level
bool | is_trained
| set if the Index does not require training, or if training is done already
MetricType | metric_type
| type of metric this index uses for search

## Additional Inherited Members
Public Types inherited from faiss::IndexBinary
typedef long | idx_t
| all indices are this type

## Detailed Description

Index based on a inverted file (IVF)

In the inverted file, the quantizer (an IndexBinary instance) provides a quantization index for each vector to be added. The quantization index maps to a list (aka inverted list or posting list), where the id of the vector is stored.

Otherwise the object is similar to the IndexIVF

Definition at line 36 of file IndexBinaryIVF.h.

## Constructor & Destructor Documentation

faiss::IndexBinaryIVF::IndexBinaryIVF | ( | IndexBinary * | quantizer,
|  | size_t | d,
|  | size_t | nlist
| ) |  |

The Inverted file takes a quantizer (an IndexBinary) on input, which implements the function mapping a vector to a list identifier. The pointer is borrowed: the quantizer should not be deleted while the IndexBinaryIVF is in use.

Definition at line 27 of file IndexBinaryIVF.cpp.

## Member Function Documentation

void faiss::IndexBinaryIVF::add_with_ids | ( | idx_t | n,
|  | const uint8_t * | x,
|  | const long * | xids
| ) |  |
| override virtual

Same as add, but stores xids instead of sequential ids.

The default implementation fails with an assertion, as it is not supported by all indexes.

Parameters

  xids | if non-null, ids to store for the vectors (size n)

Reimplemented from faiss::IndexBinary.

Definition at line 61 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::make_direct_map | ( | bool | new_maintain_direct_map = `true` | ) |

intialize a direct map

Parameters

  new_maintain_direct_map | if true, create a direct map, else clear it

Definition at line 105 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::merge_from | ( | IndexBinaryIVF & | other,
|  | idx_t | add_id
| ) |  |
| virtual

moves the entries from another dataset to self. On output, other is empty. add_id is added to all moved ids (for sequential ids, this would be this->ntotal

Definition at line 286 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::reconstruct | ( | idx_t | key,
|  | uint8_t * | recons
| ) |  | const
| override virtual

Reconstruct a stored vector.

This function may not be defined for some indexes.

Parameters

  key | id of the vector to reconstruct
  recons | reconstucted vector (size d / 8)

Reimplemented from faiss::IndexBinary.

Definition at line 141 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::reconstruct_from_offset | ( | long | list_no,
|  | long | offset,
|  | uint8_t * | recons
| ) |  | const
| virtual

Reconstruct a vector given the location in terms of (inv list index + inv list offset) instead of the id.

Useful for reconstructing when the direct_map is not maintained and the inv list offset is computed by search_preassigned() with `store_pairs` set.

Definition at line 203 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::reconstruct_n | ( | idx_t | i0,
|  | idx_t | ni,
|  | uint8_t * | recons
| ) |  | const
| override virtual

Reconstruct a subset of the indexed vectors.

Overrides default implementation to bypass reconstruct() which requires direct_map to be maintained.

Parameters

  i0 | first vector to reconstruct
  ni | nb of vectors to reconstruct
  recons | output array of reconstructed vectors, size ni * d / 8

Reimplemented from faiss::IndexBinary.

Definition at line 149 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::search | ( | idx_t | n,
|  | const uint8_t * | x,
|  | idx_t | k,
|  | int32_t * | distances,
|  | idx_t * | labels
| ) |  | const
| override virtual

assign the vectors, then call search_preassign

Implements faiss::IndexBinary.

Definition at line 128 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::search_and_reconstruct | ( | idx_t | n,
|  | const uint8_t * | x,
|  | idx_t | k,
|  | int32_t * | distances,
|  | idx_t * | labels,
|  | uint8_t * | recons
| ) |  | const
| override virtual

Similar to search, but also reconstructs the stored vectors (or an approximation in the case of lossy coding) for the search results.

Overrides default implementation to avoid having to maintain direct_map and instead fetch the code offsets through the `store_pairs` flag in search_preassigned().

Parameters

  recons | reconstructed vectors size (n, k, d / 8)

Reimplemented from faiss::IndexBinary.

Definition at line 168 of file IndexBinaryIVF.cpp.

void faiss::IndexBinaryIVF::search_preassigned | ( | idx_t | n,
|  | const uint8_t * | x,
|  | idx_t | k,
|  | const idx_t * | assign,
|  | const int32_t * | centroid_dis,
|  | int32_t * | distances,
|  | idx_t * | labels,
|  | bool | store_pairs,
|  | const IVFSearchParameters * | params = `nullptr`
| ) |  | const

Search a set of vectors, that are pre-quantized by the IVF quantizer. Fill in the corresponding heaps with the query results. search() calls this.

Parameters

  n | nb of vectors to query
  x | query vectors, size nx * d
  assign | coarse quantization indices, size nx * nprobe
  centroid_dis | distances to coarse centroids, size nx * nprobe
  distance | output distances, size n * k
  labels | output labels, size n * k
  store_pairs | store inv list index + inv list offset instead in upper/lower 32 bit of result, instead of ids (used for reranking).
  params | used to override the object's search parameters

Definition at line 662 of file IndexBinaryIVF.cpp.

## Member Data Documentation

bool faiss::IndexBinaryIVF::use_heap = true

Select between using a heap or counting to select the k smallest values when scanning inverted lists.

Definition at line 47 of file IndexBinaryIVF.h.

The documentation for this struct was generated from the following files:

- /tmp/faiss/IndexBinaryIVF.h
- /tmp/faiss/IndexBinaryIVF.cpp