Qdrant - vector similarity search engine with extended filtering support

Overview

Qdrant

Vector Similarity Search Engine with extended filtering support

Telegram

Qdrant (read: quadrant ) is a vector similarity search engine. It provides a production-ready service with a convenient API to store, search, and manage points - vectors with an additional payload. Qdrant is tailored to extended filtering support. It makes it useful for all sorts of neural-network or semantic-based matching, faceted search, and other applications.

Qdrant is written in Rust πŸ¦€ , which makes it reliable even under high load.

With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!

Demo Projects

Semantic Text Search πŸ”

The neural search uses semantic embeddings instead of keywords and works best with short texts. With Qdrant and a pre-trained neural network, you can build and deploy semantic neural search on your data in minutes. Try it online!

Similar Image Search - Food Discovery πŸ•

There are multiple ways to discover things, text search is not the only one. In the case of food, people rely more on appearance than description and ingredients. So why not let people choose their next lunch by its appearance, even if they don’t know the name of the dish? Check it out!

Extreme classification - E-commerce Product Categorization πŸ“Ί

Extreme classification is a rapidly growing research area within machine learning focusing on multi-class and multi-label problems involving an extremely large number of labels. Sometimes it is millions and tens of millions classes. The most promising way to solve this problem is to use similarity learning models. We put together a demo example of how you could approach the problem with a pre-trained transformer model and Qdrant. So you can play with it online!

More solutions
Semantic Text Search Similar Image Search Recommendations
Chat Bots Matching Engines

API

Online OpenAPI 3.0 documentation is available here. OpenAPI makes it easy to generate a client for virtually any framework or programing language.

You can also download raw OpenAPI definitions.

Features

Filtering

Qdrant supports key-value payload associated with vectors. It does not only store payload but also allows filter results based on payload values. It allows any combinations of should, must, and must_not conditions, but unlike ElasticSearch post-filtering, Qdrant guarantees all relevant vectors are retrieved.

Rich data types

Vector payload supports a large variety of data types and query conditions, including string matching, numerical ranges, geo-locations, and more. Payload filtering conditions allow you to build almost any custom business logic that should work on top of similarity matching.

Query planning and payload indexes

Using the information about the stored key-value data, the query planner decides on the best way to execute the query. For example, if the search space limited by filters is small, it is more efficient to use a full brute force than an index.

SIMD Hardware Acceleration

With the BLAS library, Qdrant can take advantage of modern CPU architectures. It allows you to search even faster on modern hardware.

Write-ahead logging

Once the service confirmed an update - it won't lose data even in case of power shut down. All operations are stored in the update journal and the latest database state could be easily reconstructed at any moment.

Stand-alone

Qdrant does not rely on any external database or orchestration controller, which makes it very easy to configure.

Usage

Docker 🐳

Build your own from source

docker build . --tag=qdrant

Or use latest pre-built image from DockerHub

docker pull generall/qdrant

To run container use command:

docker run -p 6333:6333 \
    -v $(pwd)/path/to/data:/qdrant/storage \
    -v $(pwd)/path/to/custom_config.yaml:/qdrant/config/production.yaml \
    qdrant
  • /qdrant/storage - is a place where Qdrant persists all your data. Make sure to mount it as a volume, otherwise docker will drop it with the container.
  • /qdrant/config/production.yaml - is the file with engine configuration. You can override any value from the reference config

Now Qdrant should be accessible at localhost:6333

Docs πŸ““

Contacts

Contributors ✨

Thanks to the people who contributed to Qdrant:


Andrey Vasnetsov

πŸ’»

Andre Zayarni

πŸ“–

Joan Fontanals

πŸ’»

trean

πŸ’»

Konstantin

πŸ’»

License

Qdrant is licensed under the Apache License, Version 2.0. View a copy of the License file.

Comments
  • Individual endpoints for points operation

    Individual endpoints for points operation

    Follow up on #32 which handled the collections operations.

    The goal of this issue is to discuss the design of the new update API for points.

    The current update API is:

    POST /collections/{col_name} with different commands possible:

    • PointOperations
      • "upsert_points" : array of points OR batch
      • "delete_points": array of ids
    • PayloadOps
      • "set_payload": payload object AND array of points
      • "delete_payload": keys AND array of points
      • "clear_payload": array of points
    • FieldIndexOperations
      • "create_index": index name
      • "delete_index": index name

    One possible solution is to deprecate the current endpoint and introduce 7 new specialized endpoints

    1. update points

    PUT /collections/{col_name}/points/

    • array of points OR batch as body
    1. delete points

    DELETE /collections/{col_name}/points/

    • array of ids as query parameters
    1. set payload

    POST /collections/{col_name}/points/{name}/payload

    • payload object AND array of points as body
    1. delete payload

    PUT /collections/{col_name}/points/{name}/payload

    • keys AND array of points as body
    1. clear payload

    PUT /collections/{col_name}/points/{name}/payload

    • array of ids as query parameters
    1. create index

    POST /collections/{col_name}/index/

    • index name as body
    1. delete index

    DELETE /collections/{col_name}/index/{index_name}

    to be released 
    opened by agourlay 18
  • Point search too slow

    Point search too slow

    Hi, Thank you for this amazing project. I am planning to get help in creating a vector search engine with qdrant. I created a collection like below using docker container.

    url = f"{qdrant_host}/collections/{model_name}"
            data = {
                "name": model_name,
                "distance": "Cosine",
                "vector_size": 768
            }
            response = requests.put(url,
                                    headers={'Content-Type':'application/json; charset=utf8'},
                                    data=json.dumps(data))
    

    I put points like below.

    batch = []
    for payload, embedding in data:
      """
      payload = {
                      'short_str1': str,
                      'integer_format_date_1': integer,
                      'list_of_str1': list,
                      'integer_format_date_2': integer,
                      'short_str2': str,
                      'list_of_str2': list
                  }
      """
      batch.append({
                                    'id': cnt,
                                    'payload': payload,
                                    'vector': embedding
                                })
      cnt += 1
    
    qd_url = f"{qdrant_host}/collections/{model_name}/points"
    response = requests.put(qd_url,
                                            headers={'Content-Type':'application/json; charset=utf8'},
                                            data=json.dumps(data))
    

    Loaded about 620,000 points. It takes about 10-20 seconds to search for a point as shown below.

    data = {
            'vector': embedding,
            'with_payload': True,
            'score_threshold': 0.8,
            'top': 10
        }
    qd_url = f"{qdrant_host}/collections/{model_name}/points/search"
    response = requests.post(qd_url,
                                 headers={'Content-Type':'application/json; charset=utf8'},
                                 data=json.dumps(data))
    

    I think processing time increases linearly with the number of registered vectors. Is this normal?

    opened by YongWookHa 14
  • Application hangs during shutdown

    Application hangs during shutdown

    Current Behavior

    The application hangs during shutdown with ctr+c after having received traffic from benches/service/collection_stress.js for a few seconds.

    Steps to Reproduce

    1. start application with cargo run --release --bin qdrant
    2. start HTTP benchmark
    3. ./benches/service/run.sh
    4. wait 1 minute
    5. send ctr+c to application
    6. terminal hangs after a few log lines
    [2022-01-24T15:10:46Z INFO  actix_server::server] SIGINT received; starting forced shutdown
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    [2022-01-24T15:10:46Z DEBUG actix_server::accept] Paused accepting connections on 127.0.0.1:6333
    [2022-01-24T15:10:46Z INFO  actix_server::accept] Accept thread stopped
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    [2022-01-24T15:10:46Z INFO  actix_server::worker] Shutting down idle worker
    

    Threads that are still alive:

    hang

    Expected Behavior

    The application should shutdown gracefully.

    bug wontfix 
    opened by agourlay 10
  • Test HTTP API

    Test HTTP API

    Is your feature request related to a problem? Please describe.

    The HTTP API is currently not tested which represents a risk in terms of introducing regressions.

    The only safety net present is having developers execute locally ./tests/basic_api_test.sh before submitting PRs.

    The current situation presents several issues:

    • many features are not tested via the bash script
    • it is not enforced in the development workflow
    • increasing the test coverage is not part of the development worklow
    • using a bashscript is not super ergonomic

    Describe the solution you'd like

    1. HTTP tests that are executed as part of the CI pipeline to catch regressions
    2. pick a testing strategy (see next paragraph)
    3. make HTTP testing part of the requirements to merge new features

    Describe alternatives you've considered

    Concerning the testing strategy, there are 2 main approaches.

    1. self contained tests
    • tests are spawning servers https://actix.rs/docs/testing/
    • pro: have everything contained in the core project
    • contra: Rust is not necessarily the best language to express complex tests
    1. black box testing:
    • spawn a server and execute tests written in a DSL such as cucumber or cornichon (shameless plug :sweat_smile:)
    • pro: more readable and easier for non dev to add tests
    • contra: spawing a server makes things more complicated & introducing additional non Rust tooling
    opened by agourlay 10
  • Remove blas

    Remove blas

    Remove blas from QDrant and use SIMD from std library. Works only for x86 and x64 arcs (because arm SIMD presents only in nightly rust).

    All Submissions:

    • [x] Have you followed the guidelines in our Contributing document?
    • [x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

    New Feature Submissions:

    1. [x] Does your submission pass tests?
    2. [x] Have you lint your code locally using cargo fmt command prior to submission?
    3. [x] Have you checked your code using cargo clippy command?

    Changes to Core Features:

    • [x] Have you added an explanation of what your changes do and why you'd like us to include them?
    • [x] Have you written new tests for your core changes, as applicable?
    • [x] Have you successfully ran tests with your changes locally?
    opened by IvanPleshkov 10
  • Allow to flush segment in separate thread

    Allow to flush segment in separate thread

    All Submissions:

    • [x] Have you followed the guidelines in our Contributing document?
    • [x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

    New Feature Submissions:

    1. [x] Does your submission pass tests?
    2. [x] Have you lint your code locally using cargo fmt command prior to submission?
    3. [x] Have you checked your code using cargo clippy command?

    Changes to Core Features:

    • [ ] Have you added an explanation of what your changes do and why you'd like us to include them?
    • [x] Have you written new tests for your core changes, as applicable?
    • [x] Have you successfully ran tests with your changes locally?
    opened by IvanPleshkov 9
  • [sharding] Collection cluster info API

    [sharding] Collection cluster info API

    This PR adds a new REST API to get the collection cluster info. (#624)

    With this API an operator can get extra information about the shard topology of a given collection.

    It is tested via the sharding consensus test, here is an example of output.

    edit: new format

    {
        "shard_count": 6,
        "local_shards": [
            {
                "shard_id": 2
            }
        ],
        "remote_shards": [
            {
                "shard_id": 0,
                "peer_id": 5206661873001331473
            },
            {
                "shard_id": 1,
                "peer_id": 9153846410978676289
            },
            {
                "shard_id": 3,
                "peer_id": 11473594217186723880
            },
            {
                "shard_id": 4,
                "peer_id": 17502153356522390965
            },
            {
                "shard_id": 5,
                "peer_id": 5206661873001331473
            }
        ]
    }
    
    opened by agourlay 8
  • Search query crushes container (depends on host)

    Search query crushes container (depends on host)

    On one host system, a search query crushes the container. On another with the same data set it works fine.

    Current Behavior

    After a search query, docker container crushed.

    Docker logs: docker run -p 6333:6333 -l debug -v ${pwd}/qdrant/storage:/qdrant/storage generall/qdrant:

    [2021-07-27T16:09:49Z INFO  wal::segment] Segment { path: "./storage/collections/test_collection/wal/open-5", entries: 31, space: (2424040/33554432)
    }: opened
    [2021-07-27T16:09:49Z INFO  wal::segment] Segment { path: "./storage/collections/test_collection/wal/open-6", entries: 0, space: (8/33554432) }: opened
    [2021-07-27T16:09:49Z INFO  wal::segment] Segment { path: "./storage/collections/test_collection/wal/closed-1374", entries: 418, space: (33474360/33554432) }: opened
    [2021-07-27T16:09:49Z INFO  wal] Wal { path: "./storage/collections/test_collection/wal", segment-count: 2, entries: [1374, 1823)  }: opened
    [2021-07-27T16:09:56Z INFO  qdrant] loaded collection: test_collection
    [2021-07-27T16:09:56Z INFO  actix_server::builder] Starting 12 workers
    [2021-07-27T16:09:56Z INFO  actix_server::builder] Starting "actix-web-service-0.0.0.0:6333" service on 0.0.0.0:6333
    

    Docker logs with strace: strace docker run -p 6333:6333 -l debug -v ${pwd}/qdrant/storage:/qdrant/storage generall/qdrant:

    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL[2021-07-27T16:13:27Z INFO  wal::segment] Segment { path: "./storage/collections/test_collection/wal/open-5", entries: 31, space: (2424040/33554432) }: opened
    [2021-07-27T16:13:27Z INFO  wal::segment] Segment { path: "./storage/collections/test_collection/wal/open-6", entries: 0, space: (8/33554432) }: opened
    ) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL[2021-07-27T16:13:27Z INFO  wal::segment] Segment { path: "./storage/collections/test_collection/wal/closed-1374", entries: 418, space: (33474360/33554432) }: opened
    [2021-07-27T16:13:27Z INFO  wal] Wal { path: "./storage/collections/test_collection/wal", segment-count: 2, entries: [1374, 1823)  }: opened
    ) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL[2021-07-27T16:13:35Z INFO  qdrant] loaded collection: test_collection
    [2021-07-27T16:13:35Z INFO  actix_server::builder] Starting 12 workers
    [2021-07-27T16:13:35Z INFO  actix_server::builder] Starting "actix-web-service-0.0.0.0:6333" service on 0.0.0.0:6333
    ) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
    --- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
    futex(0x55d8f6e79c40, FUTEX_WAKE_PRIVATE, 1) = 1
    rt_sigreturn({mask=[]})                 = 202
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    futex(0x55d8f6e5cce8, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
    epoll_pwait(4, [], 128, 0, NULL, 0)     = 0
    epoll_pwait(4,  <unfinished ...>)       = ?
    +++ exited with 132 +++
    

    Steps to Reproduce

    Collection like:

    {
        "create_collection": {
            "name": "test_collection",
            "vector_size": 256,
            "distance": "Cosine"
        }
    }
    

    Number of vectors does not affect the error.

    Search request like: POST /collections/test_collection/points/search

    {
        'vector': [...],
        'top': 10
    }
    

    Expected Behavior

    Stable container work and return search result. And сhecking resource availability.

    Possible Solution

    Maybe affected by processor instructions set or problems with free memory? What is the size of the free memory block for searching?

    Context (Environment)

    • Host system based on Intel(R) Core(TM) i7-3930
    • Memory free: ~1.8-2G
    • Ubuntu 18.04
    • Docker version 20.10.2
    • latest pre-built image from DockerHub
    bug 
    opened by tmnhy 8
  • [Question] is there any doc. to set distributed setup in kubernetes

    [Question] is there any doc. to set distributed setup in kubernetes

    i have seen your distributed setup using raft. i am curious to use this in kubernetes if you provide any info or doc regarding this will be helpfull thank you!

    opened by sivaguru-pat 7
  • [Sharding] Introduce cluster configuration

    [Sharding] Introduce cluster configuration

    This PR introduce a proper configuration for the distributed deployment. #495

    One decision I took is to separate the consensus configuration from the strictly p2p configuration as there are kind of orthogonal.

    The cluster deployment is still hidden behind the compile time flag --features consensus, it will be addressed later on in #511.

    opened by agourlay 6
  • Remove overcomplicated iterator

    Remove overcomplicated iterator

    All Submissions:

    • [x] Have you followed the guidelines in our Contributing document?
    • [x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

    New Feature Submissions:

    1. [x] Does your submission pass tests?
    2. [x] Have you lint your code locally using cargo fmt command prior to submission?
    3. [x] Have you checked your code using cargo clippy command?

    Changes to Core Features:

    • [ ] Have you added an explanation of what your changes do and why you'd like us to include them?
    • [x] Have you written new tests for your core changes, as applicable?
    • [x] Have you successfully ran tests with your changes locally?
    opened by IvanPleshkov 6
  • integrate quantized data to storages

    integrate quantized data to storages

    All Submissions:

    • [ ] Have you followed the guidelines in our Contributing document?
    • [ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

    New Feature Submissions:

    1. [ ] Does your submission pass tests?
    2. [ ] Have you lint your code locally using cargo fmt command prior to submission?
    3. [ ] Have you checked your code using cargo clippy command?

    Changes to Core Features:

    • [ ] Have you added an explanation of what your changes do and why you'd like us to include them?
    • [ ] Have you written new tests for your core changes, as applicable?
    • [ ] Have you successfully ran tests with your changes locally?
    opened by IvanPleshkov 0
  • Bump arc-swap from 1.5.1 to 1.6.0

    Bump arc-swap from 1.5.1 to 1.6.0

    Bumps arc-swap from 1.5.1 to 1.6.0.

    Changelog

    Sourced from arc-swap's changelog.

    1.6.0

    • Fix a data race reported by MIRI.
    • Avoid violating stacked borrows (AFAIK these are still experimental and not normative, but better safe than sorry). (#80).
    • The AccessConvert wrapper is needed less often in practice (#77).
    Commits
    • 2a62e2b Version 1.6.0
    • 41e7e7d Document the bug in two-layer option
    • 81a56de Merge pull request #87 from vorner/future-proof-rc
    • 7af4c73 Future-proof the Rc::as_ptr too
    • 634dd80 Use serde_test for unit test instead of serde_json. (#67)
    • ca4b62f Merge pull request #86 from vorner/ub-ref
    • 97a65cd Store the debt list as pointers, not refs
    • fd04e20 Future-proof as_ptr impls
    • eaeae27 Avoid stacked-borrows violations
    • 4793409 Re-enable stacked borrows in CI
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Try Advice::Random for vector storage & HNSW index mmap

    Try Advice::Random for vector storage & HNSW index mmap

    Is your feature request related to a problem? Please describe. Currently, we do not specify the pattern of mmap usege, which means, that it operates in MADV_NORMAL. Default behavior might invoke reading of multiple sequential pages, which might slow down the search in case low available RAM and\or slow disk.

    Describe the solution you'd like Make an experiment with Advice::Random ( libc::MADV_RANDOM ) on low ram usage, similar to what was described in this article: https://qdrant.tech/articles/memory-consumption/

    (use of random vectors for comparison is fine, I think)

    Describe alternatives you've considered It might be possible, that random access will only benefit the search speed if RAM << Vectors Size. If we will use mmap storage with enough ram, we are interested in faster filling of the disk cache. Decision of enabling random access advice might require gathering additional parameters.

    Additional context Some users reported, that usage of swap instead of mmaps increased the speed. Might it be related to the read strategy of swap?

    opened by generall 0
  • Accept 64bit integers as strings for javascript clients? Discuss.

    Accept 64bit integers as strings for javascript clients? Discuss.

    Is your feature request related to a problem? Please describe. I generated 64 bit integers for ids for my collection points, as strings we're not supported (related: #610 ) I was able to create the collection fine using a python client.

    When I tried to query my points using nodejs, i get the problem that i need to generate ids that are technically 'out of spec' for JSON

    Some other services (https://developers.google.com/discovery/v1/type-format#:~:text=For%20example%2C%20a%2064%2Dbit,string%20in%20JSON%20requests%2Fresponses.) allow using strings instead to side-step this problem. Recent versions of javascript support Bigint, but apparently when serializing to JSON, this also goes to a string.

    Describe the solution you'd like

    If i try to serialize the id as a string, i get this: "response": {"result":null,"status":{"error":"Json deserialize error: data did not match any variant of untagged enum ExtendedPointId at line 1 column 30"},"time":0}

    Maybe it's easy enough to coherse it to an 64 bit int on the server side?

    Describe alternatives you've considered a) Support other payload encodings besides JSON, as JSON spec does not support 64 bit integers officially. b) clearly state the limitation in the docs. (not ideal but a start)

    opened by ncoder 8
Releases(v0.11.6)
  • v0.11.6(Dec 27, 2022)

    Last release of 2022 :santa:

    Change log

    Features :ribbon:

    • https://github.com/qdrant/qdrant/pull/1283 - implement an ability to lookup vectors in different collection in recommendation API. Might be useful for item-to-user recommendations, where user and item embeddings are stored in a different collection with different index configuration.

    Improvements :sewing_needle:

    • https://github.com/qdrant/qdrant/pull/1280 - optimize shard live-cycle operations, stop optimizer faster on deletion and remove files in async task, so the API response faster, preventing timeouts even on network-mounted disks.
    • https://github.com/qdrant/qdrant/pull/1286 - Fix a list of problems, related to consensus operations, including:
      • Properly handle sync transfer cancellation
      • Replicas now have a local state, which is periodically synchronized with consensus. It allows to notify consensus of failed operations and cancelled transfers even if original notification was dropped due to leader failure.
      • Improved validation on transfer initialization - prevent creating a transfer which involved a replica, which is already being transferred
      • Fixed bug related to replica inconsistency caused by updates during transfer finish
      • Improve shards transfer workflow - prevent re-try in case if transfer was cancelled externally

    Bug fixes :t-rex:

    • https://github.com/qdrant/qdrant/pull/1281 - fix error code propagation in remote shards
    • https://github.com/qdrant/qdrant/pull/1302 - fix id mapper inconsistency in case of interrupted flush routine
    Source code(tar.gz)
    Source code(zip)
  • v0.11.5(Dec 6, 2022)

    Change log

    Features πŸš€

    • https://github.com/qdrant/qdrant/pull/1245 - The new API for payload manipulation allows you to overwrite all existing payload fields in one step. Unlike set_payload, which replaces only mentioned fields, this new API overwrites all existing fields.
    • https://github.com/qdrant/qdrant/pull/1249 - Allow to set and delete payload fields by filter. It is now possible to specify a subset of points to update using filtering conditions. Might be useful if you are using secondary key in the payload.

    Improvements πŸ› οΈ

    • https://github.com/qdrant/qdrant/pull/1262 - add parameter that defines a source of truth for snapshot recovery. Simplifies different scenarios of snapshot recovering in distributed mode. Including the recovery on the empty cluster.
    • https://github.com/qdrant/qdrant/pull/1257 - allows to disable global HNSW index in the collection and only build HNSW sub-graphs based on payload values. Useful for organizing a collection which consists of sub-groups of different sizes, so that only the search within each group is required.
    • https://github.com/qdrant/qdrant/pull/1260 - Add the peer ID into the auto-generated snapshot name to allow for the distinction of snapshots from different nodes.

    Bug Fixes πŸ›

    • https://github.com/qdrant/qdrant/pull/1252 - Removes the blocking code in RaftService that impacts the threads of the internal gRPC Tokio runtime. As a result consensus-related operations are now have significantly less timeouts under high load.
    • https://github.com/qdrant/qdrant/pull/1255 - fix warning message during batch update, if the batch have non-unique points
    Source code(tar.gz)
    Source code(zip)
  • v0.11.4(Nov 25, 2022)

    Change log

    Features :fountain:

    • https://github.com/qdrant/qdrant/pull/1211 - mmap support for HNSW graph storage. Now it is not required to keep HNSW index in RAM. Especially relevant to collections with many small vectors.
    • https://github.com/qdrant/qdrant/pull/1186 - Enable compression in gRPC protocol, useful for transferring large payloads over the slow network.

    Improvements :golf:

    • https://github.com/qdrant/qdrant/pull/1199 - Probabilistic search sampling. Significantly improves the search speed in case of large limit values. Allows to query large amount of nearest vectors fast.
    • https://github.com/qdrant/qdrant/pull/1171 - Use compact CPU cache-friendly storage of HNSW graph links. Improves search speed, memory usage and application startup time.
    • https://github.com/qdrant/qdrant/pull/1221 - improve response speed of cluster-info API

    Bug Fixes :crocodile:

    • https://github.com/qdrant/qdrant/pull/1241 - fixes possible deadlocking of consensus thread
    • https://github.com/qdrant/qdrant/pull/1238 - fixes incorrect shard activation during snapshot recovery in distributed mode
    • https://github.com/qdrant/qdrant/pull/1236 - fixes possible panic in proxy segment
    • https://github.com/qdrant/qdrant/pull/1223 - avoid service shutdown in case of removing failed collection
    • https://github.com/qdrant/qdrant/pull/1226 - prevent execution thread blocking on snapshot creation
    Source code(tar.gz)
    Source code(zip)
  • v0.11.3(Nov 15, 2022)

    Change log

    Improvements

    • https://github.com/qdrant/qdrant/pull/1214 - API for recovering snapshots, supports recovering in distributed mode

    Bug fixes

    • https://github.com/qdrant/qdrant/pull/1212 - fixed persistence of collection parameters changing
    Source code(tar.gz)
    Source code(zip)
  • v0.11.2(Nov 9, 2022)

    Change log

    Bug fixes

    • https://github.com/qdrant/qdrant/pull/1208 - fix error on collection replication factor and write consistency factor update
    • https://github.com/qdrant/qdrant/pull/1198 - handle zero-length vectors with cosine distance - skip normalization and store vector as-is, in this case if will have 0 score with all other vectors

    Improvements

    • https://github.com/qdrant/qdrant/pull/1176 - Use less ram for id tracker, especially useful for collections with many small vectors
    • https://github.com/qdrant/qdrant/pull/1184 - enable HTTP compression middleware
    • https://github.com/qdrant/qdrant/pull/1200 - ensures read-after-write consistency in case of replica failure and if requests are going to the same peer.
    Source code(tar.gz)
    Source code(zip)
  • v0.11.1(Nov 5, 2022)

    Changelog

    Improvements

    • https://github.com/qdrant/qdrant/pull/1189 - handle collections transition from single-node to cluster. Simplifies migration from single-node deployment into cluster version. Also allows to recover single-node version snapshots into cluster deployment
    Source code(tar.gz)
    Source code(zip)
  • v0.11.0(Oct 26, 2022)

    Changelog

    Features :cake:

    • Replication support - Now Qdrant can provide a high availability setup with distributed deployment out of the box. Replications in combination with sharding allow horizontally scaling of both - the collection's size and the cluster's throughput. While at the same time eliminating a single point of failure problem. Read more in the docs :book:
    • https://github.com/qdrant/qdrant/issues/1119 - New administration API allows disabling write operations to the service. Useful in situations where search availability is more critical than updates. It will enable blocking updates if, e.g., the memory usage watermark is reached. Docs :book:
    • https://github.com/qdrant/qdrant/pull/1164 - Report indexed payload points in info API, allows to verify that payload values was properly formatted for indexing
    • https://github.com/qdrant/qdrant/pull/1150 - New exact search parameter allows to force exact search of the vectors, even if ANN index is built. Useful for validating accuracy of the current HNSW configuration.

    Improvements :seedling:

    • https://github.com/qdrant/qdrant/pull/1157 - Disable logging for healthchecks
    • https://github.com/qdrant/qdrant/pull/1152 - Move connection errors logging into /cluster status API. Prevents flood of error messages in case of other node failure in cluster

    Bug fixes :imp:

    • https://github.com/qdrant/qdrant/pull/1172 - fix issue with consensus WAL, leading to panic on recover in some rare cases

    Compatibility :dancers:

    • This release is backward compatible with v0.10.5 storage in single node deployment.
    • Distributed deployment is, unfortunately, incompatible with previous versions due to the large amount of changes, required for replica set implementation.
    • Clients are tested for backward compatibility with v0.10.x service

    Further work :astronaut:

    This release finalizes the feature-set of the first road-map of Qdrant. Further updates before v1.0 will be focused on maintainability and stability of the service. The next road-map will be published soon!

    Thanks to @IvanPleshkov, @agourlay, and @e-ivkov for contributing into this release

    Source code(tar.gz)
    Source code(zip)
  • v0.10.5(Oct 18, 2022)

  • v0.10.4(Oct 13, 2022)

  • v0.10.3(Oct 11, 2022)

    Changelog

    Bug fixes

    • https://github.com/qdrant/qdrant/pull/1108 - fix conflict resolution in Raft consensus WAL. Consensus is able to recover after split-brain cluster
    • https://github.com/qdrant/qdrant/pull/1109 - fix panic on inconsistent storage updates. Now storage is sound after error recovery
    • https://github.com/qdrant/qdrant/pull/1110 - additional guarantees on storage consistency in case of multiple vector collection
    • https://github.com/qdrant/qdrant/pull/1102 - more explicit leader election flag in the Raft consensus
    • https://github.com/qdrant/qdrant/pull/1083 - properly handle a corner-case of limit = 0 in search API
    Source code(tar.gz)
    Source code(zip)
  • v0.10.2(Sep 30, 2022)

    Changelog

    Bug fixes

    • https://github.com/qdrant/qdrant/pull/1041 - propagate proper error in case of partial update error across shards
    • https://github.com/qdrant/qdrant/pull/1050 - fix the filter-based points remove operation
    • https://github.com/qdrant/qdrant/pull/1054 - ensure storage consistency in case of failure during collection creation
    • https://github.com/qdrant/qdrant/pull/1061 - increase stability during high load
    Source code(tar.gz)
    Source code(zip)
  • v0.10.1(Sep 19, 2022)

    Backward compatibility release

    See: https://github.com/qdrant/qdrant/releases/tag/v0.10.0

    Proposed update process:

    1. Update qdrant from v0.9.1 -> v0.10.0
    2. Update qdrant client to v0.10.0
    3. Update qdrant to v0.10.1
    4. Update qdrant client to v0.10.1

    | Version compatibility | client v0.9.x | client v0.10.0 | client v0.10.1 | |-------------------------|---------------|----------------|----------------| | qdrant v0.9.1 | + | - | - | | qdrant v0.10.0 | + | + | | | qdrant v0.10.1 | - | + | + |

    Source code(tar.gz)
    Source code(zip)
  • v0.10.0(Sep 19, 2022)

    Changelog

    Features :ferris_wheel:

    • https://github.com/qdrant/qdrant/pull/958 - Multiple Vectors per Point. - Docs
      • It is now possible to assign multiple vectors for each point in collection. Update and search APIs now accepts either one anonymous vector (same as before), or multiple named vectors. This functionality might be useful if some object could be represented from multiple aspects by different neural encoders (e.g. image + text, title + abstract).
    PUT /collections/{collection_name}/points
    
    {
        "points": [
            {
                "id": 1,
                "vectors": {
                    "image": [0.9, 0.1, 0.1, 0.2],
                    "text": [0.4, 0.7, 0.1, 0.8, 0.1, 0.1, 0.9, 0.2]
                }
            }
       ]
    }
    
    • https://github.com/qdrant/qdrant/pull/813 & https://github.com/qdrant/qdrant/pull/954 - Batch Search and Recommendation API - Docs
      • Allows more efficient search for multiple queries in a single request. Significantly decreases networking overhead and shares the result of filtering computation among batched queries.
    POST /collections/{collection_name}/points/search/batch
    
    {
        "searches": [
            { "vector": [0.2, 0.1, 0.9, 0.7], "limit": 3 },
            { "vector": [0.5, 0.3, 0.2, 0.3], "limit": 3 }
        ]
    }
    
    • https://github.com/qdrant/qdrant/pull/963 - Full-Text Filtering - Docs
      • Adds minimal full-text search capabilities into all filterable APIs. Specify ngram-prefix or word tokenizer to make queries only among records which contain specified words.

    Improvements :mortar_board:

    • https://github.com/qdrant/qdrant/pull/1000 - Multi-platform builds for x86_64 and aarch64 - allows natively run Qdrant on arm-based CPUs. Pushes price/performance ratio even further. - DockerHub
    • https://github.com/qdrant/qdrant/pull/949 - Use Jemallocator instead of default one. Significantly improves memory usage, prevents heap fragmentation. - Read more
    • https://github.com/qdrant/qdrant/pull/1003 - Improve scroll API performance for strict filters. In some cases up to 1000x speed improvement.

    image

    Bug fixes :space_invader:

    • https://github.com/qdrant/qdrant/pull/973 - fix for snapshot api, do not fail if FS does not support last-update timestamp
    • https://github.com/qdrant/qdrant/pull/990 - fix telemetry API (not stabilized yet)
    • https://github.com/qdrant/qdrant/pull/1023 - assign proper response code on non-specified payload index type

    Compatibility :electric_plug:

    There are a lot of fundamental changes in qdrant segment storage and API, but we did our best to make a transition process as smooth as possible. The v0.10 release be performed in 2 steps: on the first step we upgrade qdrant into transition version v0.10.0 which support both - old and new version of the interface. The second update v0.10.1 will remove deprecated fields from the API.

    Proposed update process with minimal downtime:

    1. Update qdrant from v0.9.1 -> v0.10.0
    2. Update qdrant client to v0.10.0
    3. Update qdrant to v0.10.1
    4. Update qdrant client to v0.10.1

    | Version compatibility | client v0.9.x | client v0.10.0 | client v0.10.1 | |-------------------------|---------------|----------------|----------------| | qdrant v0.9.1 | + | - | - | | qdrant v0.10.0 | + | + | | | qdrant v0.10.1 | - | + | + |

    Don't forget to make backups and test the process in development environment!

    See API changes in client releases:

    Further work :hammer:

    Next release will be primarily focused on the Replication Support.

    Thanks to @IvanPleshkov, @agourlay, @e-ivkov, @joein and @monatis for contributing into this release

    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Aug 10, 2022)

    Compatibility fix release

    Fixes

    • https://github.com/qdrant/qdrant/pull/937 - Fix max_request_size_mb configuration
    • https://github.com/qdrant/qdrant/pull/936 - Do not panic if config.optimizer_config.max_optimization_threads is 0
    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Aug 8, 2022)

    Changelog

    Features :genie:

    • Dynamic cluster scaling capabilities - docs
      • https://github.com/qdrant/qdrant/issues/861 - Move shard API
      • https://github.com/qdrant/qdrant/pull/894 - Peer removal API

    Improvements :person_fencing:

    • https://github.com/qdrant/qdrant/pull/900 - Remove temporary snapshot files during full snapshot
    • https://github.com/qdrant/qdrant/pull/877 - Disable default mmap threshold
    • https://github.com/qdrant/qdrant/pull/901 - Remove default max optimizer threads to enable switching off the optimizer
    • https://github.com/qdrant/qdrant/pull/879 - Report progress of indexing with indexed_vectors_count field in the Collection info API

    Bug fixes :worm:

    • https://github.com/qdrant/qdrant/pull/876 - Handle properly collection with vector dimension of 0
    • https://github.com/qdrant/qdrant/pull/898 - Fix various enum conversion in gRPC

    Further work :construction:

    The next major release will focus on finalizing the Roadmap and bringing frequently requested functionality into the Qdrant. The next major release will be 0.10.0. After that, we will work on stabilizing the API and storage as preparation for a 1.0 LTS release.

    Thanks @agourlay & @e-ivkov for contributing into this release!

    Source code(tar.gz)
    Source code(zip)
  • v0.8.6(Jul 26, 2022)

  • v0.8.5(Jul 26, 2022)

    Change log

    Features :blueberries:

    • https://github.com/qdrant/qdrant/pull/824 - Full storage snapshot - creates a snapshot of all collections and aliases in one call

    Improvements :factory:

    • https://github.com/qdrant/qdrant/pull/783 - Parallel segment loading - significantly improves service recovery time

    • Several distributed deployment stability improvements:

      • https://github.com/qdrant/qdrant/pull/798 - better defaults for internal calls timeout
      • https://github.com/qdrant/qdrant/pull/797 - propagation of wait flag during distributed calls
      • https://github.com/qdrant/qdrant/pull/799 - automatic selection of number of shards
      • https://github.com/qdrant/qdrant/pull/816 - more even point distribution across shards
      • https://github.com/qdrant/qdrant/pull/822 - API for viewing collection shards distribution
      • https://github.com/qdrant/qdrant/pull/851 - better handling of DNS record changes of peers
    • https://github.com/qdrant/qdrant/pull/821 - fix backward compatibility with storages created in 0.8.x

    Bug fixes :shrimp:

    • https://github.com/qdrant/qdrant/pull/849 - fix possible inconsistency in payload index in case of inserting multiple payloads with the same value

    Further work

    The main goal for v0.9.0 is a seamless cluster scaling - ability to add more nodes into the cluster and move shards between this nodes.

    Thanks @agourlay, @e-ivkov, and @IvanPleshkov for contributing into this release!

    Source code(tar.gz)
    Source code(zip)
  • v0.8.4(Jul 5, 2022)

    Change log

    Features :hatching_chick:

    • Snapshots - https://github.com/qdrant/qdrant/pull/764 https://github.com/qdrant/qdrant/pull/772

      • Allows to safely create a snapshot of running collection and recover it on another machine
      • Useful for backups, tests, and ensuring high availability
      • Documentation: https://qdrant.tech/documentation/snapshots/
    • Count API - https://github.com/qdrant/qdrant/pull/777

      • Allows to retrieve a number of points that satisfy given filtering criteria
      • Useful for: pagination, facet search, debugging
      • Documentation: https://qdrant.tech/documentation/points/#counting-points

    Improvements :eagle:

    • Parallel building of HNSW index - https://github.com/qdrant/qdrant/commit/e983b07a1521cd47771b63006defe54f74d181ce
      • Qdrant allows now to utilize multiple processes for building HNSW index.
      • Building of large HNSW segments now benefits from the multi-core systems while significantly decreasing overall index building time.
    • Improved condition for creating additional links in HNSW graph. https://github.com/qdrant/qdrant/pull/773
      • Additional links for payload with low variety of values won't be created as they do not have an impact on search performance.
    • Improved behavior of Raft consensus during the bootstrap retry - https://github.com/qdrant/qdrant/issues/768

    Bug fixes :octopus:

    • fix db flush ordering to prevent persisting versions for un-persisted points https://github.com/qdrant/qdrant/commit/8ac3eef4c7e1519d78c0036b779b192cab482c94

    Thanks @agourlay @e-ivkov for contributing to the release

    Source code(tar.gz)
    Source code(zip)
  • v0.8.3(Jun 28, 2022)

    Change log

    Features

    • Pagination of search result: https://github.com/qdrant/qdrant/issues/496

    Bug Fixes

    • Batch upsert with distributed deployment - https://github.com/qdrant/qdrant/issues/758
    • Stability improvements: https://github.com/qdrant/qdrant/commit/8ac3eef4c7e1519d78c0036b779b192cab482c94
    Source code(tar.gz)
    Source code(zip)
  • v0.8.2(Jun 24, 2022)

    Change log - Stability improvements

    Bug fixes

    • Fixed possible deadlock on collection read https://github.com/qdrant/qdrant/pull/725, thanks @BurtonQin
    • Stability patch for distributed deployment: https://github.com/qdrant/qdrant/pull/710, https://github.com/qdrant/qdrant/pull/737, https://github.com/qdrant/qdrant/pull/712
    • Fixes in concurrent storage update & optimization: https://github.com/qdrant/qdrant/pull/736
    Source code(tar.gz)
    Source code(zip)
  • v0.8.1(Jun 16, 2022)

    Change log

    Improvements

    • New web-based UI :art: available: https://ui.qdrant.tech/, make requests to local qdrant instance right from the browser

    Bug fixes

    • Fixed bug related to directory removal in case of using volumes mounted with CIFS. Issues: https://github.com/qdrant/qdrant/issues/701 and https://github.com/qdrant/qdrant/issues/308 PR: https://github.com/qdrant/qdrant/pull/705
    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Jun 8, 2022)

    Change log

    :crystal_ball: Features

    • Experimental support for distributed deployment - [docs] [issues]

      • Cluster topology synchronization via Raft consensus protocol
      • Sharding for the collections
      • Distributed search and updates
      • Cluster status API - https://github.com/qdrant/qdrant/issues/605
    • Filtering by similarity score - https://github.com/qdrant/qdrant/issues/113

    • On-disk payload storage - https://github.com/qdrant/qdrant/issues/406

      • Store large payload with less RAM usage
    • On-flight payload indexing - payload indexes are available without segment re-build

      • Streaming updates for Numeric, Keyword, and Geo indexes: https://github.com/qdrant/qdrant/issues/575, https://github.com/qdrant/qdrant/issues/592, https://github.com/qdrant/qdrant/issues/593

    :wrench: Improvements

    • Faster payload index - https://github.com/qdrant/qdrant/pull/461

    • Better error reporting

      • Return errors as JSON https://github.com/qdrant/qdrant/issues/555
      • Better handling of mismatched enums https://github.com/qdrant/qdrant/issues/551
    • Enable CORS headers for Swagger UI access - https://github.com/qdrant/qdrant/issues/610

    • HNSW Speed improvements (https://github.com/qdrant/qdrant/pull/508, https://github.com/qdrant/qdrant/pull/513)

    • Storage optimization

      • RocksDB better log rotation - https://github.com/qdrant/qdrant/issues/516
      • Disable redundant WAL in RocksDB - https://github.com/qdrant/qdrant/pull/576
      • Unite RocksDB columns in single DB - https://github.com/qdrant/qdrant/pull/585
      • Parallel segment creation - https://github.com/qdrant/qdrant/pull/598
    • Better support for Int payload in gRPC - https://github.com/qdrant/qdrant/pull/564

    :construction: Breaking changes

    • Due to significant changes in payload storage, collections created with the previous engine version are not compatible with the new version. The most straightforward migration approach would be to re-create collections with the latest version from scratch. Breaking changes are required to make our v1.0 release as legacy-free as possible. Hope for your understanding.

    :hourglass: Work-in-progress

    • We are continuing the development of Distributed Deployment. In next release we will address shard re-balancing and replication issues. Stay tuned and check out our Roadmap for details.
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Apr 13, 2022)

    Change log

    :carousel_horse: Features

    • Qdrant now supports any JSON as payload - https://github.com/qdrant/qdrant/pull/306

      • The uploaded payload structure now exactly matches the retrieved results.
      • The payload schema is now obsolete. The payload type is now used for field indexes only.
    • Alias API in gRPC - Making up for a lost API https://github.com/qdrant/qdrant/pull/408

    • New filtering condition

      • Bool filter and refactoring of Match - https://github.com/qdrant/qdrant/pull/421
      • IsEmpty filter - https://github.com/qdrant/qdrant/pull/423
      • ValuesCount filter - https://github.com/qdrant/qdrant/pull/439

    :rocket: Improvements

    • Geo payload indexing - https://github.com/qdrant/qdrant/pull/366

      • Geolocation queries are now processed with a special type of index.
    • Removed BLAS - https://github.com/qdrant/qdrant/pull/182

      • Custom, slim, architecture-aware implementations now perform distance computation.
      • Additional support for neon aarch64.
      • Euclidean metric now uses SIMD - order of magnitude faster performance.
    • Use Filtering context instead of Condition Checker - https://github.com/qdrant/qdrant/pull/413

      • Opens up opportunities for additional optimization during the planning phase of the queries.
    • sled removed - https://github.com/qdrant/qdrant/pull/402

      • Less overhead resources.
    • HNSW Performance improvements

      • Better iteration implementation - https://github.com/qdrant/qdrant/pull/409
      • Chunked Vector storage - https://github.com/qdrant/qdrant/pull/457

    :heavy_exclamation_mark: Breaking changes

    • Due to significant changes in payload format, collections created with the previous engine version are not compatible with the new version. The most straightforward migration approach would be to re-create collections with the latest version from scratch. Breaking changes are required to make our v1.0 release as legacy-free as possible. Hope for your understanding.

    :microscope: Work-in-progress

    • Our main efforts are now focused on the distributed deployment feature. Stay tuned a check out our Roadmap for details.
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Mar 4, 2022)

    Change log

    Features

    • gPRC support for all endpoints
      • Protobug docs: https://github.com/qdrant/qdrant/blob/master/docs/grpc/docs.md
      • Instruction how to enable: https://qdrant.tech/documentation/quick_start/#grpc
    • recommend endpoint can now return payload and vector - https://github.com/qdrant/qdrant/issues/353

    Bug fixes

    • Better logic for segment flushing, prevent accumulating of the large amount of data in WAL - https://github.com/qdrant/qdrant/pull/336
    • Fixed payload selector in all retrieval APIs https://github.com/qdrant/qdrant/pull/358

    Breaking changes

    • REST API is compatible, but if you are using python client, please follow the migration examples here https://github.com/qdrant/qdrant_client/releases/tag/v0.6.0
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Feb 16, 2022)

    Change log

    Features

    • Better error reporting in collection info API: https://github.com/qdrant/qdrant/pull/316
    • [geo] validate coordinates in API - https://github.com/qdrant/qdrant/issues/294

    Bug fixes

    • optimizer parameters update bug: https://github.com/qdrant/qdrant/issues/309
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Feb 3, 2022)

    Change log

    Features

    • Multi-threaded optimizer - https://github.com/qdrant/qdrant/issues/30
    • Allow to use UUID as point ID - https://github.com/qdrant/qdrant/pull/265
    • Include payload and vector into search result https://github.com/qdrant/qdrant/issues/50
    • Delete points by filter - https://github.com/qdrant/qdrant/issues/39
    • Remove payload using filters - https://github.com/qdrant/qdrant/issues/269
    • Individual endpoints for collection & point operations https://github.com/qdrant/qdrant/issues/32
    • Limit segment size for better indexing performance - https://github.com/qdrant/qdrant/issues/135

    Bug fixes

    • Collection remove times-out while optimization is in process - https://github.com/qdrant/qdrant/issues/31

    Deprecation

    • Update Collection and Update Points APIs are deprecated. Please use individual endpoints for each type of update operations instead. E.g. use https://qdrant.github.io/qdrant/redoc/index.html?v=master#operation/set_payload to set payload

    Breaking changes

    • Due to changes related to UUID and configuration format, indexes created with the previous version of the engine are not compatible with the new version. The most straightforward migration approach would be to re-create collections with the latest version from scratch. Breaking changes are required to make our v1.0 release as legacy-free as possible. Hope for your understanding.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Oct 27, 2021)

  • v0.4.1(Oct 25, 2021)

    Change log

    Bug Fixes

    • Alias creation issue fixed https://github.com/qdrant/qdrant/issues/103
    • Proper return code for operations with non-exiting collections https://github.com/qdrant/qdrant/issues/99
    • Proper (this time) use of DYNAMIC_ARCH for support older CPU architecture as well as modern SIMD instructions in BLAS https://github.com/qdrant/qdrant/issues/78
    • Fixed https://github.com/qdrant/qdrant/issues/77 - issue related to possible corruption of storage. Related changes:
      • Now all updates go through a dedicated thread to ensure proper update sequence
      • Each record now have it's individual version - to ensure nothing is mixed during a segment swap
      • Segments now have a failure lock - in case of failed update operation (e.g. not enough disk space) the failed segment will be locked for write until failed operation is (automatically) re-applied

    Features

    • It is now possible to retrieve payload along with search query in a single request - https://github.com/qdrant/qdrant/issues/50 Also field selection/exclusion works
    Source code(tar.gz)
    Source code(zip)
  • v0.3.6(Aug 29, 2021)

    Change log

    Bug Fixes

    • Fix point retrieve from copy-on-write proxy segment - https://github.com/qdrant/qdrant/pull/94 Related issue - https://github.com/qdrant/qdrant/issues/80
    • Deadlock on parallel segment holder access - https://github.com/qdrant/qdrant/pull/91 Related issue - https://github.com/qdrant/qdrant/issues/75
    • Handle corner case of limit = 0 in scroll API - https://github.com/qdrant/qdrant/issues/90
    • Use DYNAMIC_ARCH=1 for OpenBLAS build. Related issue - https://github.com/qdrant/qdrant/issues/78
    Source code(tar.gz)
    Source code(zip)
  • v0.3.5(Jul 11, 2021)

    Change log

    Improvements

    • Actix upgraded to version 4.0beta, which allows to completely upgrade Tokio to 1.0+

    Bug fix

    • WAL now uses CBOR format, which properly handles some corner cases of payload interface format. The implementation is backward compatible
    Source code(tar.gz)
    Source code(zip)
Owner
qdrant
Vector search project
qdrant
Kalman filtering and smoothing in Rust

Kalman filtering and smoothing library written in Rust Access documentation for the library here. Library is also referenced in Cargo index. Currently

Rytis Bagdziunas 27 Nov 2, 2022
Rust implementation of user-based collaborative filtering

Rucommender Recommendation system written in Rust Overview An implementation in Rust of a collaborative filtering recommendations algorithm with a use

null 0 Sep 15, 2018
Rust implementation of @Qdrant/fastembed.

FastEmbed-rs ?? Rust implementation of @Qdrant/fastembed ?? Features Supports synchronous usage. No dependency on Tokio. Uses @huggingface/tokenizers

Anush 35 Oct 30, 2023
FFSVM stands for "Really Fast Support Vector Machine"

In One Sentence You trained a SVM using libSVM, now you want the highest possible performance during (real-time) classification, like games or VR. Hig

Ralf Biedert 53 Nov 24, 2022
Rust port of the extended isolation forest algorithm for anomaly detection

Extended Isolation Forest This is a rust port of the anomaly detection algorithm described in Extended Isolation Forest and implemented in https://git

Nico Mandery 6 Oct 21, 2022
Embedded Rust arithmetic, 2D/3D vector, and statistics library

Embedded-friendly (i.e. no_std) Rust math library featuring fast, safe floating point approximations for common arithmetic operations, trigonometry, 2D/3D vector types, statistical analysis, and quaternions.

Tony Arcieri 318 Dec 22, 2022
A vector with a fixed capacity. (Rust)

arrayvec OR A vector with fixed capacity. Please read the API documentation here License Dual-licensed to be compatible with the Rust project. License

bluss 522 Dec 20, 2022
🚧 WIP 🚧 Vector database plugin for Postgres, written in Rust, specifically designed for LLM.

pgvecto.rs pgvecto.rs is a Postgres extension that provides vector similarity search functions. It is written in Rust and based on pgrx. Features cosi

TensorChord 74 Apr 26, 2023
HNSW ANN from the paper "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs"

hnsw Hierarchical Navigable Small World Graph for fast ANN search Enable the serde feature to serialize and deserialize HNSW. Tips A good default for

Rust Computer Vision 93 Dec 30, 2022
Graph-based Approximate Nearest Neighbor Search

granne* granne (graph-based retrieval of approximate nearest neighbors) is a Rust library for approximate nearest neighbor search based on Hierarchica

null 283 Dec 21, 2022
πŸš€ efficient approximate nearest neighbor search algorithm collections library written in Rust πŸ¦€ .

?? efficient approximate nearest neighbor search algorithm collections library written in Rust ?? .

Hora-Search 2.3k Jan 3, 2023
Nearest neighbor search algorithms including a ball tree and a vantage point tree.

petal-neighbors Nearest neighbor search algorithms including a ball tree and a vantage point tree. Examples The following example shows how to find tw

Petabi 6 Oct 19, 2022
Hamming Weight Tree from the paper Online Nearest Neighbor Search in Hamming Space

hwt Hamming Weight Tree from the paper Online Nearest Neighbor Search in Hamming Space To understand how the data structure works, please see the docs

Rust Computer Vision 7 Oct 9, 2021
A neural network model that can approximate any non-linear function by using the random search algorithm for the optimization of the loss function.

random_search A neural network model that can approximate any non-linear function by using the random search algorithm for the optimization of the los

ph04 2 Apr 1, 2022
Believe in AI democratization. llama for nodejs backed by llama-rs, work locally on your laptop CPU. support llama/alpaca model.

llama-node Large Language Model LLaMA on node.js This project is in an early stage, the API for nodejs may change in the future, use it with caution.

Genkagaku.GPT 145 Apr 10, 2023
Rust crate for reconstructing Arabic sentences to be used in applications that don't support Arabic

Arabic Reshaper Rust Reconstruct Arabic sentences to be used in applications that don't support Arabic script. Usage: resahpe a single line of string

YouKnow 5 Jun 21, 2023
The Hacker's Machine Learning Engine

Juice This is the workspace project for juice - machine learning frameworks for hackers coaster - underlying math abstraction coaster-nn coaster-blas

spearow 982 Dec 31, 2022
Personal experiments with genetic algorithms and neuroevolution, in Rust, with the Bevy engine.

The Tango Problem Personal experiments with genetic algorithms and neuroevolution, in Rust, with the Bevy engine. A number of "Psychics" are placed in

null 3 Nov 20, 2023
The Fast Vector Similarity Library is designed to provide efficient computation of various similarity measures between vectors.

Fast Vector Similarity Library Introduction The Fast Vector Similarity Library is designed to provide efficient computation of various similarity meas

Jeff Emanuel 243 Sep 6, 2023
Machine learning framework for building object trackers and similarity search engines

Similari Similari is a framework that helps build sophisticated tracking systems. The most frequently met operations that can be efficiently implement

In-Sight 71 Dec 28, 2022